Oct 30 21:01:12.351612 ip-10-0-139-2 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:01:12.708193 ip-10-0-139-2 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:01:12.708193 ip-10-0-139-2 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:01:12.708193 ip-10-0-139-2 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:01:12.708193 ip-10-0-139-2 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:01:12.715585 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715518    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.715585 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715584    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.715585 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715587    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.715585 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715590    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715593    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715596    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715599    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715601    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715604    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715607    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715610    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715612    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715615    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715617    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715620    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715623    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715626    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715629    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715632    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715634    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715637    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715639    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715642    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.715728 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715646    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715651    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715654    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715657    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715659    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715662    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715665    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715667    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715673    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715676    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715679    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715681    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715684    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715687    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715690    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715693    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715696    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715700    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715704    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715707    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.716208 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715710    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715713    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715716    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715718    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715721    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715724    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715726    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715729    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715731    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715734    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715737    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715740    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715743    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715745    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715748    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715751    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715753    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715757    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715759    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.716694 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715762    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715764    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715767    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715770    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715772    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715775    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715778    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715782    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715784    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715787    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715789    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715792    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715795    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715797    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715800    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715802    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715805    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715807    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715810    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715812    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.717147 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715815    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715819    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715821    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715824    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715827    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715829    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715832    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715834    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715837    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715839    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715842    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715845    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715847    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715850    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715853    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715855    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715858    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715860    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715863    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715866    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.717630 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715869    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715871    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715874    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.715878    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717786    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717794    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717798    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717801    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717804    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717806    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717809    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717812    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717814    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717817    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717821    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717823    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717826    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717828    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717831    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717834    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.718074 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717837    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717840    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717842    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717845    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717848    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717851    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717855    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717858    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717861    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717864    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717867    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717869    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717872    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717875    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717878    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717881    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717884    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717886    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717889    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717891    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.718552 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717894    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717896    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717899    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717901    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717906    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717910    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717913    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717916    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717919    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717922    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717925    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717927    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717930    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717932    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717935    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717937    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717940    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717942    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717945    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.719125 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717948    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717950    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717953    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717955    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717957    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717960    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717963    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717966    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717969    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717971    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717974    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717976    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717979    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717981    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717984    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717986    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717989    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717991    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717994    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717996    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.719567 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.717999    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718002    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718004    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718007    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718010    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718012    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718014    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718017    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718019    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718022    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718024    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718027    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718029    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718032    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718034    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718037    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718039    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718043    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718046    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.720061 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718049    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718052    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718054    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718057    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718059    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718061    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718064    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718066    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718069    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718072    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718074    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718077    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718580    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718586    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718590    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718593    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718595    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718598    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718601    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718603    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.720524 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718606    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718608    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718611    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718614    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718616    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718620    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718624    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718627    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718629    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718632    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718635    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718638    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718641    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718644    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718647    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718650    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718653    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718655    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718658    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.720987 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718661    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718663    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718666    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718668    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718671    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718674    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718676    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718679    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718682    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718684    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718687    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718690    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718692    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718695    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718697    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718700    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718703    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718706    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718708    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.721421 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718711    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718713    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718716    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718719    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718722    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718724    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718727    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718730    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718732    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718735    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718737    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718740    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718742    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718745    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718747    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718750    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718753    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718756    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718758    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718761    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.721881 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718763    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718766    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718769    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718772    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718774    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718777    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718780    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718782    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718785    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718787    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718790    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718792    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718795    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718798    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718800    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718803    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718805    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718808    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718810    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718813    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.722334 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718815    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718818    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718820    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718823    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718826    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718828    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718831    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718833    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718836    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718840    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718844    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718847    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718850    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718852    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718855    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718863    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718866    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718868    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718871    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.718873    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.722800 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719538    2579 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719546    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719552    2579 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719556    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719560    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719563    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719568    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719574    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719577    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719580    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719584    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719587    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719590    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719593    2579 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719596    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719599    2579 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719602    2579 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719605    2579 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719611    2579 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719613    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719617    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719620    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719623    2579 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719631    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:01:12.723253 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719634    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719637    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719640    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719644    2579 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719648    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719651    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719654    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719657    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719661    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719664    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719667    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719670    2579 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719672    2579 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719675    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719679    2579 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719684    2579 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719686    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719689    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719692    2579 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719696    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719699    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719702    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719705    2579 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719708    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719726    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:01:12.723860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719730    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719734    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719737    2579 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719740    2579 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719743    2579 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719747    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719750    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719753    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719756    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719759    2579 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719762    2579 flags.go:64] FLAG: --help="false"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719765    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719769    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719772    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719775    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719778    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719784    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719787    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719790    2579 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719793    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719796    2579 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719799    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719804    2579 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719807    2579 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:01:12.724497 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719810    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719813    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719816    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719819    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719822    2579 flags.go:64] FLAG: --lock-file=""
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719825    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719827    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719830    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719836    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719839    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719842    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719845    2579 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719848    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719851    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719854    2579 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719857    2579 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719861    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719864    2579 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719868    2579 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719871    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719874    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719877    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719880    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719883    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719887    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:01:12.725060 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719890    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719897    2579 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719900    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719903    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719906    2579 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719910    2579 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719914    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719917    2579 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719919    2579 flags.go:64] FLAG: --port="10250"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719922    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719925    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-03a103c138a44637b"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719929    2579 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719932    2579 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719934    2579 flags.go:64] FLAG: --register-node="true"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719937    2579 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719941    2579 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719944    2579 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719947    2579 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719949    2579 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719953    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719956    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719959    2579 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719962    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719964    2579 flags.go:64] FLAG: --runonce="false"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719967    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719970    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719973    2579 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:01:12.725742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719976    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719979    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719982    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719985    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719989    2579 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719992    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719995    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.719998    2579 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720001    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720004    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720008    2579 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720011    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720016    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720019    2579 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720021    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720025    2579 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720028    2579 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720031    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720034    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720036    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720039    2579 flags.go:64] FLAG: --v="2"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720043    2579 flags.go:64] FLAG: --version="false"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720047    2579 flags.go:64] FLAG: --vmodule=""
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720055    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720058    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:01:12.726359 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720189    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720194    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720197    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720200    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720203    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720206    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720208    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720211    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720213    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720216    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720223    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720227    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720230    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720233    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720236    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720239    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720243    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720248    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.727029 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720252    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720255    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720257    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720260    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720262    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720265    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720268    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720270    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720273    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720276    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720278    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720281    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720283    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720285    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720288    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720290    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720293    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720295    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720298    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720300    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.727493 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720303    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720305    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720308    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720310    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720313    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720317    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720320    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720322    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720325    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720327    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720330    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720333    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720335    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720338    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720340    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720343    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720346    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720348    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720352    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.727948 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720355    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720359    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720362    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720365    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720367    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720370    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720372    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720375    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720377    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720380    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720382    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720385    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720387    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720390    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720392    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720395    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720397    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720400    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720403    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720407    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.728415 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720409    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720412    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720414    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720417    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720419    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720422    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720452    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720456    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720458    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720461    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720463    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720466    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720468    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720471    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720474    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720476    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720479    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720481    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720484    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720487    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.728899 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720489    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720492    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720494    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720497    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720499    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720502    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720504    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720507    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720509    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.720536    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720575    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720579    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720583    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720586    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720589    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.729360 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720592    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720596    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720600    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720603    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720606    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720609    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720612    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720615    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720618    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720621    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720623    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720626    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720629    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720631    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720634    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720637    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720639    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720642    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720645    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720647    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.729731 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720650    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720652    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720654    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720657    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720660    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720662    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720665    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720669    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720671    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720674    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720677    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720679    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720683    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720686    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720688    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720691    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720694    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720696    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720699    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720702    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.730222 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720704    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720707    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720710    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720714    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720717    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720720    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720723    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720726    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720728    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720731    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720733    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720736    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720739    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720741    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720744    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720746    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720749    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720751    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720754    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.730705 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720758    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720760    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720763    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720766    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720768    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720773    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720776    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720779    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720781    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720784    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720787    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720789    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720792    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720794    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720797    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720799    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720802    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720805    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720807    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.731151 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720810    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720812    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720815    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720818    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720820    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720823    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720825    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720828    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720830    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720833    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720836    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720839    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720841    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720845    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720848    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720850    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720853    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720855    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720860    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720862    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.731595 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720865    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720867    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.720870    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.726798    2579 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.726812    2579 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.726829    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.726833    2579 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726886    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726891    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726895    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726898    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726901    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726904    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726906    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726910    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726913    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726916    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726919    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726922    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.732062 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726924    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726927    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726930    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726932    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726935    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726938    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726940    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726944    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726947    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726949    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726952    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726956    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726960    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726964    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726967    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726970    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726972    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726976    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726978    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726981    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.732516 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726983    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726986    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726988    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726991    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726993    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726996    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.726998    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727001    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727003    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727006    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727008    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727011    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727013    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727016    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727019    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727021    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727024    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727026    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727029    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.732983 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727033    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727035    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727038    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727041    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727043    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727046    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727049    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727051    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727054    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727056    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727059    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727066    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727069    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727071    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727074    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727076    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727079    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727082    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727084    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.733428 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727087    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727089    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727092    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727094    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727097    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727099    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727102    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727104    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727107    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727109    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727112    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727115    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727118    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727121    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727123    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727126    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727129    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727131    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727134    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727137    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.734039 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727139    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727141    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727144    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727146    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727149    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727152    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727154    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727157    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727160    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727162    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727165    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727167    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727169    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727172    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727174    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727177    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.734748 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.727204    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727336    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727341    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727344    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727346    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727349    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727352    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727354    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727357    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727360    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727362    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727365    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727367    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727370    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727372    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727375    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727377    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727379    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727382    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727384    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.735316 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727387    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727389    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727392    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727395    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727397    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727400    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727403    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727405    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727407    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727410    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727412    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727415    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727417    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727420    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727422    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727424    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727427    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727429    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727449    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.735847 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727452    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727455    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727457    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727460    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727462    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727465    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727468    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727470    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727473    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727475    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727478    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727480    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727482    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727486    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727490    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727493    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727497    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727500    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727503    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.736314 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727506    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727509    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727511    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727514    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727516    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727519    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727521    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727524    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727526    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727529    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727531    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727534    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727537    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727539    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727542    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727544    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727547    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727550    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727552    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.737099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727554    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727557    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727559    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727562    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727564    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727567    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727569    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727572    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727574    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727576    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727579    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727582    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727584    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727587    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727589    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727592    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727594    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727597    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727599    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727602    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.737890 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727604    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727607    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727609    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727611    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727614    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727618    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727621    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727624    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727626    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727629    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.727652    2579 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727682    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727685    2579 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727689    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727692    2579 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727694    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:01:12.738348 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727697    2579 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727700    2579 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727702    2579 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727705    2579 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727707    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727710    2579 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727712    2579 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727715    2579 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727718    2579 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727720    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727723    2579 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727725    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727728    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727730    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727733    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727735    2579 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727738    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727742    2579 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727745    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727747    2579 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:01:12.738779 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727750    2579 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727752    2579 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727754    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727757    2579 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727759    2579 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727762    2579 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727764    2579 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727767    2579 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727769    2579 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727772    2579 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727774    2579 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727776    2579 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727779    2579 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727782    2579 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727784    2579 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727786    2579 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727789    2579 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727791    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727794    2579 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727796    2579 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:01:12.739255 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727799    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727802    2579 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727804    2579 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727807    2579 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727809    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727811    2579 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727814    2579 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727817    2579 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727821    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727829    2579 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727832    2579 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727834    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727837    2579 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727840    2579 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727843    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727845    2579 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727849    2579 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727852    2579 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727854    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:01:12.739759 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727857    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727860    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727863    2579 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727865    2579 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727868    2579 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727870    2579 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727873    2579 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727876    2579 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727878    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727881    2579 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727883    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727885    2579 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727888    2579 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727890    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727893    2579 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727899    2579 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727901    2579 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727904    2579 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727906    2579 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:01:12.740304 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727909    2579 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727911    2579 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727914    2579 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727918    2579 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727920    2579 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727923    2579 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727925    2579 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727928    2579 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727930    2579 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727933    2579 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727937    2579 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727940    2579 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727942    2579 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727945    2579 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727948    2579 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727951    2579 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727954    2579 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727956    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727959    2579 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:01:12.740837 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727964    2579 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727968    2579 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727971    2579 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:12.727974    2579 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.728022    2579 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.729677    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.730458    2579 server.go:1014] "Starting client certificate rotation"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.730554    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.730589    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.732685    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:01:12.741272 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.737857    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:01:12.746650 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.746634    2579 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:01:12.749601 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.749582    2579 log.go:25] "Validated CRI v1 image API"
Oct 30 21:01:12.750333 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.750312    2579 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:01:12.752675 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.752656    2579 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ec605715-8ce4-4785-aaeb-765a8204f407:/dev/nvme0n1p3 fd7421f5-b59d-41d1-ac59-a2e8e87156df:/dev/nvme0n1p4]
Oct 30 21:01:12.752766 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.752674    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 21:01:12.757546 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.757424    2579 manager.go:217] Machine: {Timestamp:2026-10-30 21:01:12.756128085 +0000 UTC m=+0.276716201 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101738 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ae5838e695c1881ab41fb4c9d9ca2 SystemUUID:ec2ae583-8e69-5c18-81ab-41fb4c9d9ca2 BootID:3528c481-b330-4707-bcd4-c54c62f1d00d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ee:71:9f:8d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ee:71:9f:8d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:b5:03:be:e5:17 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:01:12.757619 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.757548    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:01:12.757650 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.757636    2579 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:01:12.760543 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.760504    2579 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:01:12.760675 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.760544    2579 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-2.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:01:12.761154 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.761144    2579 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:01:12.761188 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.761156    2579 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:01:12.761188 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.761182    2579 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:01:12.761243 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.761194    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:01:12.761798 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.761788    2579 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:01:12.765017 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765005    2579 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:01:12.765118 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765109    2579 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:01:12.765773 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765764    2579 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:01:12.765801 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765777    2579 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:01:12.765801 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765786    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:01:12.765801 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765794    2579 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:01:12.765901 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.765801    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:01:12.766115 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.766096    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:01:12.771624 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.771605    2579 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:01:12.772989 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.772970    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:01:12.775321 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.775307    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:01:12.776063 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776049    2579 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:01:12.776115 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776081    2579 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:01:12.776247 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776237    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:01:12.776277 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776252    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:01:12.776277 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776258    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:01:12.776277 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776264    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:01:12.776277 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776270    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:01:12.776277 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776276    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:01:12.776665 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776650    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:01:12.776665 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776664    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:01:12.776794 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776674    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:01:12.776794 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776683    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:01:12.776794 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776695    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:01:12.776794 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.776708    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:01:12.777739 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.777726    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:01:12.777801 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.777743    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:01:12.781068 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.781052    2579 server.go:1263] "Started kubelet"
Oct 30 21:01:12.781153 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.781127    2579 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:01:12.781219 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.781183    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:01:12.781265 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.781232    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:01:12.782123 ip-10-0-139-2 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:01:12.782558 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.782397    2579 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:01:12.784830 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.783354    2579 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:01:12.786135 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.786116    2579 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 21:01:12.786372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.786351    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-2.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 21:01:12.786710 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.786690    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-2.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 21:01:12.786769 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.786721    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 21:01:12.786920 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.786908    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:01:12.787310 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.786488    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-2.us-east-2.compute.internal.18e36a684a6777fa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-2.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-2.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 21:01:12.781027322 +0000 UTC m=+0.301615437,LastTimestamp:2026-10-30 21:01:12.781027322 +0000 UTC m=+0.301615437,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-2.us-east-2.compute.internal,}"
Oct 30 21:01:12.787710 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.787697    2579 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:01:12.787882 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.787861    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:12.788234 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788217    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:01:12.788234 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788222    2579 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:01:12.788364 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788240    2579 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:01:12.788404 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788377    2579 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:01:12.788404 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788386    2579 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:01:12.788647 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788634    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:01:12.788690 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788648    2579 factory.go:55] Registering systemd factory
Oct 30 21:01:12.788690 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788655    2579 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:01:12.788853 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788842    2579 factory.go:153] Registering CRI-O factory
Oct 30 21:01:12.788886 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788856    2579 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:01:12.788886 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788868    2579 factory.go:103] Registering Raw factory
Oct 30 21:01:12.788886 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.788877    2579 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:01:12.789582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.789571    2579 manager.go:319] Starting recovery of all containers
Oct 30 21:01:12.790667 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.790639    2579 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-2.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 21:01:12.793742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.793649    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtmtd"
Oct 30 21:01:12.794366 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.794341    2579 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 21:01:12.798689 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.798667    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xtmtd"
Oct 30 21:01:12.800092 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.800072    2579 manager.go:324] Recovery completed
Oct 30 21:01:12.805382 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.805369    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:12.807247 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807229    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:12.807315 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807264    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:12.807315 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807274    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:12.807810 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807798    2579 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:01:12.807810 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807807    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:12.807896 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807809    2579 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:01:12.807896 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.807843    2579 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:01:12.808455 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.808418    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:12.808524 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.808470    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:12.808524 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.808484    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:12.809780 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.809767    2579 policy_none.go:50] "Start"
Oct 30 21:01:12.809780 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.809782    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:01:12.809855 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.809793    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:01:12.809855 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.809768    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-2.us-east-2.compute.internal.18e36a684bf79355  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-2.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-2.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-2.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 21:01:12.807248725 +0000 UTC m=+0.327836841,LastTimestamp:2026-10-30 21:01:12.807248725 +0000 UTC m=+0.327836841,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-2.us-east-2.compute.internal,}"
Oct 30 21:01:12.811818 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.811808    2579 policy_none.go:44] "Start"
Oct 30 21:01:12.846046 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.845934    2579 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:01:12.849845 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.849834    2579 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:01:12.849921 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.849860    2579 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:01:12.849921 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.849868    2579 server.go:87] "Starting device plugin registration server"
Oct 30 21:01:12.850039 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.850029    2579 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:01:12.850085 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.850039    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:01:12.850159 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.850136    2579 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:01:12.850790 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.850210    2579 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:01:12.850790 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.850221    2579 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:01:12.850790 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.850654    2579 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:01:12.850790 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.850702    2579 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:12.950886 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.950858    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:12.952873 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.951645    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:12.952873 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.951670    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:12.952873 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.951681    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:12.952873 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.951710    2579 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:12.957789 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.957772    2579 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:12.957889 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.957797    2579 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-2.us-east-2.compute.internal\": node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:12.961318 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.961261    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:01:12.962582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.962566    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:01:12.962681 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.962590    2579 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:01:12.962681 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.962611    2579 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:01:12.962681 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.962647    2579 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 21:01:12.971568 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:12.971552    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:12.979733 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:12.979717    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.062890 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.062859    2579 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"]
Oct 30 21:01:13.062973 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.062918    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:13.065117 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.065105    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:13.065193 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.065131    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:13.065193 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.065145    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:13.067372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.067359    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:13.067529 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.067516    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.067569 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.067543    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:13.068115 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068095    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:13.068191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068116    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:13.068191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068139    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:13.068191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068150    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:13.068191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068122    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:13.068332 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.068204    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:13.070733 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.070718    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.070809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.070740    2579 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:01:13.071470 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.071453    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:01:13.071573 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.071481    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:01:13.071573 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.071495    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:01:13.080173 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.080157    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.089429 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.089414    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.089517 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.089449    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.089517 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.089472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.093699 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.093680    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.096137 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.096119    2579 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.180600 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.180576    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.190496 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.190582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190509    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.190582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190518    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.190582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.190582 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190567    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.190713 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.190567    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.281123 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.281082    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.381681 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.381653    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.397096 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.397081    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.399260 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.399239    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.482273 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.482248    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.582772 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.582724    2579 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-2.us-east-2.compute.internal\" not found"
Oct 30 21:01:13.652482 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.652463    2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:13.688445 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.688410    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.699013 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.698997    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:01:13.700543 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.700532    2579 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:01:13.713309 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.711979    2579 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:01:13.730000 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.729979    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 21:01:13.730109 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.730092    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:01:13.730150 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.730135    2579 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 21:01:13.766667 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.766645    2579 apiserver.go:52] "Watching apiserver"
Oct 30 21:01:13.783594 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.783575    2579 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:01:13.783853 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.783839    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vzxd4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn","openshift-cluster-node-tuning-operator/tuned-pkswx","openshift-image-registry/node-ca-qkbt2","openshift-network-operator/iptables-alerter-f9fz4","kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal","openshift-dns/node-resolver-6gb6m","openshift-multus/network-metrics-daemon-ctlvw","openshift-network-diagnostics/network-check-target-nkrzz","kube-system/global-pull-secret-syncer-26bcq","openshift-ovn-kubernetes/ovnkube-node-c45gr","kube-system/konnectivity-agent-m474j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal","openshift-multus/multus-2zxxc"]
Oct 30 21:01:13.786211 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.786192    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.787266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.787248    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 21:01:13.788264 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.788247    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.789772 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.789756    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.790012 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.789988    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.790100 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.790041    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:01:13.790100 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.790044    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:01:13.790100 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.790047    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:01:13.790333 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.790320    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.791021 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.791009    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.791213 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.791201    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.791699 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.791686    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hspc2\""
Oct 30 21:01:13.792137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.792125    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:01:13.792205 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.792194    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qfb4q\""
Oct 30 21:01:13.792486 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.792473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:13.794521 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794506    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.794597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.794597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794578    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:13.794696 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794615    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.794696 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794784 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794735    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794784 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794759    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.794784 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794778    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.794892 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794795    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794892 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794817    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794892 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794892 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794883    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794998 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794910    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794998 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794998 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794961    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.794998 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.794989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5zfs\" (UniqueName: \"kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.795106 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.795106 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.795106 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.795106 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.795219 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795131    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5h67\" (UniqueName: \"kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.795219 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.795219 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795188    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.795219 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.795326 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.795326 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795252    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.795326 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795276    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.795326 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795298    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.795326 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.795314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8stgg\" (UniqueName: \"kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.796643 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.796629    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:13.796963 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.796945    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.798738 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.798723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:13.798809 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.798791    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:13.800901 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.800888    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:13.800946 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.800937    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:13.802860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.802839    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:56:13 +0000 UTC" deadline="2028-04-16 15:51:45.808237068 +0000 UTC"
Oct 30 21:01:13.802913 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.802861    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12810h50m32.005378946s"
Oct 30 21:01:13.802948 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.802910    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:13.802996 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:13.802970    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:13.803788 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.803769    2579 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 21:01:13.805160 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.805138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.807372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.807354    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:13.811580 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.809990    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.811580 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.810468    2579 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:13.817593 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.817576    2579 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:13.837921 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.837903    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddp4w"
Oct 30 21:01:13.852503 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.852483    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ddp4w"
Oct 30 21:01:13.889076 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.889061    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:01:13.889980 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.889963    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.890101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890019    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.890101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890042    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mwc48\""
Oct 30 21:01:13.890101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890058    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.890101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890064    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.890244 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890192    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:01:13.890299 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890251    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:01:13.890299 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890255    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:01:13.890424 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890342    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:01:13.890424 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890360    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:01:13.890424 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.890337    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.891394 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891377    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:01:13.891559 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891536    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:01:13.891633 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891565    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.891633 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891570    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.891633 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891576    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-99mk5\""
Oct 30 21:01:13.891633 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891612    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g68zh\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891669    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x8hl8\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891693    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891743    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qbtrh\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891748    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v7745\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891751    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:01:13.891839 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.891763    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:01:13.892774 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.892745    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-wx578\""
Oct 30 21:01:13.892851 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.892804    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:01:13.895472 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895452    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:13.895564 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895509    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:13.895607 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895565    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.895607 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.895699 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895640    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.895699 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5h67\" (UniqueName: \"kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.895795 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:13.895845 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895824    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:13.895896 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895846    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.895896 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895862    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2zwb\" (UniqueName: \"kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:13.895896 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895890    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.896032 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.896032 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.896032 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.895979    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.896032 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.896164 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896034    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.896164 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896072    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.896164 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896076    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.896164 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896155    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.896361 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.896361 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896233    2579 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:01:13.896361 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.896361 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896349    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896359    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwp8v\" (UniqueName: \"kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896505    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896561    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896587    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:13.896629 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896613    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:13.897084 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896645    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897084 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896957    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.897186 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.896673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.897186 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:13.897282 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897211    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897282 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897250    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897379 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897379 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897290    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.897379 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897303    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897379 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897331    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897379 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897369    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897398    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897424    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5zfs\" (UniqueName: \"kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897478    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897517    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897541    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897569    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.897644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897775    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897842    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897867    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.897972 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897906    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.897971    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898054    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898080    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phdsl\" (UniqueName: \"kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898233    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.898256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898236    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898356    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898399    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898414    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898462    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8stgg\" (UniqueName: \"kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898511    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898563    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898601    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898670    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhc74\" (UniqueName: \"kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898699    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898733    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.898770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898872    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898874    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898917    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898942    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898964    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.898989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899047    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899057    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899133    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899168    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899179    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899209    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899266    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.899305 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899297    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899320    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47dpt\" (UniqueName: \"kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899348    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899383    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jcnw\" (UniqueName: \"kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899460    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899484    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899561    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899630    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.899785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.899727    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:13.900950 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.900931    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.908096 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.908070    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5h67\" (UniqueName: \"kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:13.910790 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.910768    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5zfs\" (UniqueName: \"kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:13.910975 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.910955    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8stgg\" (UniqueName: \"kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:13.999956 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.999924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:13.999956 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.999959    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:13.999983    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000031    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000091    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phdsl\" (UniqueName: \"kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000102    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000118    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.000138    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000201    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.000215 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.000222    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:14.5002003 +0000 UTC m=+2.020788448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000227    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000277    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000304    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhc74\" (UniqueName: \"kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000334    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000356    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000421    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000469    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000494    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000508    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000516    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000468    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000543    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47dpt\" (UniqueName: \"kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000569    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.000785 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000576    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000592    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jcnw\" (UniqueName: \"kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000606    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000662    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000667    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000724    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000723    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000730    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000759    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000767    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000787    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000830    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000860    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000885    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000902    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.001625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000937    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.000955    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.000943    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.001041    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:14.501023613 +0000 UTC m=+2.021611716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001073    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2zwb\" (UniqueName: \"kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001158    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001177    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001182    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwp8v\" (UniqueName: \"kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001214    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001231    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001252    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001257    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.002597 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001304    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001358    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001383    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001408    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001493    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001499    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001554    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001567    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001602    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001629    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001653    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001678    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001678    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001720    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.003415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001679    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001728    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001720    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001765    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001791    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001807    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001818    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001836    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001834    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001893    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001957    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.001977    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.002195    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.002254    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.004137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.003610    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:14.004718 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.004162    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.011257 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.011234    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:14.011257 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.011260    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:14.011486 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.011276    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:14.011486 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.011326    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:14.511311644 +0000 UTC m=+2.031899764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:14.012853 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.012825    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47dpt\" (UniqueName: \"kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.013257 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.013229    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdsl\" (UniqueName: \"kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.013344 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.013265    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jcnw\" (UniqueName: \"kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.013577 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.013559    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhc74\" (UniqueName: \"kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.013735 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.013716    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2zwb\" (UniqueName: \"kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.013887 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.013874    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwp8v\" (UniqueName: \"kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:14.053298 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.053257    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96f6cddab271c4d36b89844c0672b86.slice/crio-2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e WatchSource:0}: Error finding container 2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e: Status 404 returned error can't find the container with id 2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e
Oct 30 21:01:14.058575 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.058561    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:01:14.063495 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.063478    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8b94293ba553932730b321b7e106581.slice/crio-49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36 WatchSource:0}: Error finding container 49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36: Status 404 returned error can't find the container with id 49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36
Oct 30 21:01:14.127153 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.127135    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:01:14.136172 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.136152    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9a963b_963f_4195_9041_faa91154dc88.slice/crio-681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6 WatchSource:0}: Error finding container 681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6: Status 404 returned error can't find the container with id 681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6
Oct 30 21:01:14.139227 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.139212    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:01:14.146638 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.146623    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6a6b03c_0628_4bea_bbaa_66f0c0108dd7.slice/crio-e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2 WatchSource:0}: Error finding container e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2: Status 404 returned error can't find the container with id e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2
Oct 30 21:01:14.152046 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.152031    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:01:14.159072 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.159047    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb36fe40_e2d8_41ef_a19b_74f2e7c69cbc.slice/crio-215c1c652349679099331ef1956271010a19d9b483e540efb3354700820c3aab WatchSource:0}: Error finding container 215c1c652349679099331ef1956271010a19d9b483e540efb3354700820c3aab: Status 404 returned error can't find the container with id 215c1c652349679099331ef1956271010a19d9b483e540efb3354700820c3aab
Oct 30 21:01:14.166011 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.165996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:01:14.173659 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.173641    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d19a8f3_e037_4087_8d58_f1e05facef69.slice/crio-ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd WatchSource:0}: Error finding container ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd: Status 404 returned error can't find the container with id ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd
Oct 30 21:01:14.177999 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.177982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:01:14.185090 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.185073    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef54bb28_3d2f_4bc3_ae74_ded96d8dbbc6.slice/crio-7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72 WatchSource:0}: Error finding container 7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72: Status 404 returned error can't find the container with id 7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72
Oct 30 21:01:14.196813 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.196798    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:01:14.205099 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.205081    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2020321_9a3d_4786_b6cb_f00ad50011be.slice/crio-7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970 WatchSource:0}: Error finding container 7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970: Status 404 returned error can't find the container with id 7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970
Oct 30 21:01:14.231874 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.231857    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:14.235056 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.235041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2zxxc"
Oct 30 21:01:14.241340 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.241314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:14.241578 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.241560    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60535067_d426_468e_ab28_84675493491d.slice/crio-152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62 WatchSource:0}: Error finding container 152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62: Status 404 returned error can't find the container with id 152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62
Oct 30 21:01:14.243479 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.243461    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode86b21ac_fc11_4f42_a3de_db0bda5e47f6.slice/crio-09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0 WatchSource:0}: Error finding container 09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0: Status 404 returned error can't find the container with id 09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0
Oct 30 21:01:14.250500 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:14.250482    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda209dc6c_34cd_4fd3_90f2_2591f28988a8.slice/crio-3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237 WatchSource:0}: Error finding container 3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237: Status 404 returned error can't find the container with id 3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237
Oct 30 21:01:14.252419 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.252401    2579 kuberuntime_manager.go:1707] "Unhandled Error" err="container ovn-controller start failed in pod ovnkube-node-c45gr_openshift-ovn-kubernetes(a209dc6c-34cd-4fd3-90f2-2591f28988a8): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Oct 30 21:01:14.254285 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.254266    2579 kuberuntime_manager.go:1707] "Unhandled Error" err="container kube-rbac-proxy-node start failed in pod ovnkube-node-c45gr_openshift-ovn-kubernetes(a209dc6c-34cd-4fd3-90f2-2591f28988a8): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Oct 30 21:01:14.258881 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.258821    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" podUID="a209dc6c-34cd-4fd3-90f2-2591f28988a8"
Oct 30 21:01:14.504678 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.504612    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:14.504678 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.504668    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.504840 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.504755    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:14.504840 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.504757    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:14.504840 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.504816    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:15.504797975 +0000 UTC m=+3.025386094 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:14.504840 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.504830    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:15.504824478 +0000 UTC m=+3.025412581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:14.605062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.604932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:14.605220 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.605131    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:14.605220 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.605157    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:14.605220 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.605171    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:14.605342 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.605230    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:15.605215814 +0000 UTC m=+3.125803930 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:14.859724 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.855790    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:56:14 +0000 UTC" deadline="2028-05-25 23:02:05.869527362 +0000 UTC"
Oct 30 21:01:14.859724 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.855853    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13754h0m51.013678803s"
Oct 30 21:01:14.966521 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.965663    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:14.966521 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.965807    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:14.966521 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.966215    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:14.966521 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:14.966317    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:14.988849 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.988792    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237"}
Oct 30 21:01:14.995625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:14.995349    2579 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:15.004997 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.004966    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6gb6m" event={"ID":"c2020321-9a3d-4786-b6cb-f00ad50011be","Type":"ContainerStarted","Data":"7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970"}
Oct 30 21:01:15.014803 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.014702    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9fz4" event={"ID":"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6","Type":"ContainerStarted","Data":"7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72"}
Oct 30 21:01:15.016862 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.016832    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pkswx" event={"ID":"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc","Type":"ContainerStarted","Data":"215c1c652349679099331ef1956271010a19d9b483e540efb3354700820c3aab"}
Oct 30 21:01:15.025567 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.025319    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"ovn-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovn-acl-logging\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-node\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"kube-rbac-proxy-ovn-metrics\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d017b7d199b1d54e889bb1eb6e572d8e6d94ad92610894009ee0562c2a5c0f51\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"northd\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"nbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"sbdb\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"ovnkube-controller\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:be9d8303ae4f79715ade971ddecdf6044ee6bb7b8d8b3a2f2f1e3bfe4e78c15a\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" podUID="a209dc6c-34cd-4fd3-90f2-2591f28988a8"
Oct 30 21:01:15.026618 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.026108    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn" event={"ID":"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7","Type":"ContainerStarted","Data":"e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2"}
Oct 30 21:01:15.031980 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.031952    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2zxxc" event={"ID":"e86b21ac-fc11-4f42-a3de-db0bda5e47f6","Type":"ContainerStarted","Data":"09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0"}
Oct 30 21:01:15.036709 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.036682    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m474j" event={"ID":"60535067-d426-468e-ab28-84675493491d","Type":"ContainerStarted","Data":"152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62"}
Oct 30 21:01:15.040729 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.040702    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkbt2" event={"ID":"4d19a8f3-e037-4087-8d58-f1e05facef69","Type":"ContainerStarted","Data":"ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd"}
Oct 30 21:01:15.043822 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.043793    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerStarted","Data":"681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6"}
Oct 30 21:01:15.049238 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.049210    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerStarted","Data":"49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36"}
Oct 30 21:01:15.051285 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.051238    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"f96f6cddab271c4d36b89844c0672b86","Type":"ContainerStarted","Data":"2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e"}
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.512273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.512357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.512525    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.512603    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:17.512582625 +0000 UTC m=+5.033170737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.513037    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:15.513177 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.513092    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:17.513075039 +0000 UTC m=+5.033663144 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:15.612926 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.612886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:15.613119 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.613103    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:15.613213 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.613122    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:15.613213 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.613135    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:15.613213 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.613211    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:17.613196906 +0000 UTC m=+5.133785009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:15.856479 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.856349    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:56:14 +0000 UTC" deadline="2028-07-21 22:04:33.797967942 +0000 UTC"
Oct 30 21:01:15.856479 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.856387    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15121h3m17.94158572s"
Oct 30 21:01:15.964193 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:15.963659    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:15.964193 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:15.963793    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:16.809059 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:16.808973    2579 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:01:16.965757 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:16.965729    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:16.966122 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:16.965874    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:16.966273 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:16.966249    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:16.966405 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:16.966354    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:17.528062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:17.527781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:17.528062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:17.527852    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:17.528062 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.527983    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:17.528062 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.528047    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:21.528027875 +0000 UTC m=+9.048616005 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:17.528546 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.528442    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:17.528546 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.528517    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:21.528481051 +0000 UTC m=+9.049069172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:17.629106 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:17.629070    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:17.629264 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.629241    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:17.629264 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.629260    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:17.629378 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.629275    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:17.629436 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.629414    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:21.629385913 +0000 UTC m=+9.149974034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:17.963410 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:17.963229    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:17.963410 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:17.963344    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:18.072891 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.072816    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="6e7570be09a79e284f54226ee2763a17c34034798b1da1c8b8917607b231080a" exitCode=0
Oct 30 21:01:18.073314 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.072904    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"6e7570be09a79e284f54226ee2763a17c34034798b1da1c8b8917607b231080a"}
Oct 30 21:01:18.079179 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.079138    2579 generic.go:356] "Generic (PLEG): container finished" podID="e8b94293ba553932730b321b7e106581" containerID="bd734250ab4eeb48da9160bb9058e91ee69e8aca840c798f3410404e0edbe789" exitCode=0
Oct 30 21:01:18.079279 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.079186    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerDied","Data":"bd734250ab4eeb48da9160bb9058e91ee69e8aca840c798f3410404e0edbe789"}
Oct 30 21:01:18.965765 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.965744    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:18.965956 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:18.965906    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:18.966363 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:18.966344    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:18.966546 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:18.966503    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:19.083627 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.083559    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-m474j" event={"ID":"60535067-d426-468e-ab28-84675493491d","Type":"ContainerStarted","Data":"db65a1222527875a11f763cc96465c4058e7ae77ae7cabc56a174768713747d3"}
Oct 30 21:01:19.088861 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.088818    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkbt2" event={"ID":"4d19a8f3-e037-4087-8d58-f1e05facef69","Type":"ContainerStarted","Data":"2f22323bf704de4ae20e6f7de1051aa46c782c5e3f9d409189fa9274b904a43b"}
Oct 30 21:01:19.099922 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.099861    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-m474j" podStartSLOduration=2.0941555530000002 podStartE2EDuration="6.099844823s" totalImagesPullingTime="4.00568927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:19.099206476 +0000 UTC m=+6.619794602" watchObservedRunningTime="2026-10-30 21:01:19.099844823 +0000 UTC m=+6.620432950"
Oct 30 21:01:19.105607 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.105378    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerStarted","Data":"0cca92fffc3c5e3c81db6614fddc99479c8eca8bcc7d0bc580cd40f5c72554eb"}
Oct 30 21:01:19.108831 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.108808    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"f96f6cddab271c4d36b89844c0672b86","Type":"ContainerStarted","Data":"e01094799f58b9357f97d53c21880a290a7e556b3c2ee3edc317e19d939ffb11"}
Oct 30 21:01:19.128005 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.127962    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qkbt2" podStartSLOduration=1.496999865 podStartE2EDuration="6.127946715s" totalImagesPullingTime="4.63094685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:19.113656836 +0000 UTC m=+6.634244974" watchObservedRunningTime="2026-10-30 21:01:19.127946715 +0000 UTC m=+6.648534841"
Oct 30 21:01:19.147263 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.147222    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" podStartSLOduration=6.147212604 podStartE2EDuration="6.147212604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:19.128563423 +0000 UTC m=+6.649151549" watchObservedRunningTime="2026-10-30 21:01:19.147212604 +0000 UTC m=+6.667800758"
Oct 30 21:01:19.147624 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.147597    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal" podStartSLOduration=6.147590429 podStartE2EDuration="6.147590429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:19.147135889 +0000 UTC m=+6.667724017" watchObservedRunningTime="2026-10-30 21:01:19.147590429 +0000 UTC m=+6.668178554"
Oct 30 21:01:19.471008 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.470963    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:19.471712 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.471691    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:19.778372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.778348    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:19.963527 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:19.963372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:19.963527 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:19.963504    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:20.113926 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.113895    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn" event={"ID":"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7","Type":"ContainerStarted","Data":"0c13cc99882e049ccae4300dc83faa3d4d7524a5db8a6a31611378e662e7b896"}
Oct 30 21:01:20.114918 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.114898    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:20.115780 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.115602    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:01:20.226187 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.226155    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.332784 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.332753    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.586964 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.586856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.691125 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.691074    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:20.965482 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.965448    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:20.965663 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:20.965561    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:20.965886 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:20.965867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:20.965964 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:20.965940    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:21.405373 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.405350    2579 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:01:21.563348 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.563302    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:21.563478 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.563381    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:21.563536 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.563478    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:21.563536 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.563503    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:21.563649 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.563542    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:29.563524025 +0000 UTC m=+17.084112130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:21.563649 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.563562    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:29.563552717 +0000 UTC m=+17.084140829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:21.664061 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.664021    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:21.664205 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.664187    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:21.664290 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.664214    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:21.664290 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.664226    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:21.664290 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.664288    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:29.664269417 +0000 UTC m=+17.184857534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:21.853279 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.853081    2579 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:01:21.405367503Z","UUID":"3bf1a8eb-cf7d-4c14-9a81-ff25195ad2e8","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:01:21.854616 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.854596    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:01:21.854727 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.854623    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:01:21.962872 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:21.962843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:21.963020 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:21.962951    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:22.119997 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.119959    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-6gb6m" event={"ID":"c2020321-9a3d-4786-b6cb-f00ad50011be","Type":"ContainerStarted","Data":"f686a467883cf17b59fc875406ad228f448e94dac7cf22f1be314edb19a14d61"}
Oct 30 21:01:22.122289 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.122259    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn" event={"ID":"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7","Type":"ContainerStarted","Data":"ba338332d0a7b89134ef0918279b69ef7322affb5fd37399606c5f13f0d184fc"}
Oct 30 21:01:22.124117 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.124077    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2zxxc" event={"ID":"e86b21ac-fc11-4f42-a3de-db0bda5e47f6","Type":"ContainerStarted","Data":"c3ae56178a36814dfd047324c91e4107adee0fa57da086d0a70b14e9317b37b8"}
Oct 30 21:01:22.126462 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.126081    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="04a14c1949e883f3760fe09aebb3eaf8ead24b837c50bc1d9dfe2b9e030297b6" exitCode=0
Oct 30 21:01:22.126462 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.126181    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"04a14c1949e883f3760fe09aebb3eaf8ead24b837c50bc1d9dfe2b9e030297b6"}
Oct 30 21:01:22.135471 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.135411    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-6gb6m" podStartSLOduration=1.752070056 podStartE2EDuration="9.135397393s" totalImagesPullingTime="7.383327337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:22.135365531 +0000 UTC m=+9.655953660" watchObservedRunningTime="2026-10-30 21:01:22.135397393 +0000 UTC m=+9.655985514"
Oct 30 21:01:22.175540 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.175492    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-2zxxc" podStartSLOduration=2.042355559 podStartE2EDuration="9.175478143s" totalImagesPullingTime="7.133122584s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:22.174816763 +0000 UTC m=+9.695404890" watchObservedRunningTime="2026-10-30 21:01:22.175478143 +0000 UTC m=+9.696066268"
Oct 30 21:01:22.250989 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.250795    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:22.413715 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.413681    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:22.560398 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.560356    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:22.681306 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.681204    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:22.965556 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.965491    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:22.965692 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:22.965644    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:22.966746 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:22.966071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:22.966746 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:22.966193    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:23.020672 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.020646    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:23.129136 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.129090    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pkswx" event={"ID":"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc","Type":"ContainerStarted","Data":"e2c11277c83c6e0f20044bbc55ea8a0e5094fc0e43d793f8bdef8082d8826b3f"}
Oct 30 21:01:23.131147 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.131116    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn" event={"ID":"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7","Type":"ContainerStarted","Data":"4eed45b6c06207b163be4555e990e8ea91406b83e20a438085f6936abba8d190"}
Oct 30 21:01:23.133242 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.133221    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="c5d1546496764a438bef9b01c74f6a9949b2fa55e86f94de678a06afb7aa753b" exitCode=0
Oct 30 21:01:23.133333 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.133269    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"c5d1546496764a438bef9b01c74f6a9949b2fa55e86f94de678a06afb7aa753b"}
Oct 30 21:01:23.134770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.134754    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9fz4" event={"ID":"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6","Type":"ContainerStarted","Data":"36db4ea42e23e54dc870b4140ce6fa109d9ed1552a87fdec5137cb33b0cc3389"}
Oct 30 21:01:23.146473 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.146416    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pkswx" podStartSLOduration=1.316672139 podStartE2EDuration="10.146407374s" totalImagesPullingTime="8.829735235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:23.146139639 +0000 UTC m=+10.666727763" watchObservedRunningTime="2026-10-30 21:01:23.146407374 +0000 UTC m=+10.666995499"
Oct 30 21:01:23.185540 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.185509    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn" podStartSLOduration=2.345793187 podStartE2EDuration="10.185498222s" totalImagesPullingTime="7.839705035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:23.184967682 +0000 UTC m=+10.705555807" watchObservedRunningTime="2026-10-30 21:01:23.185498222 +0000 UTC m=+10.706086346"
Oct 30 21:01:23.200646 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.200616    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f9fz4" podStartSLOduration=3.09705205 podStartE2EDuration="10.200605614s" totalImagesPullingTime="7.103553564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:23.200239133 +0000 UTC m=+10.720827258" watchObservedRunningTime="2026-10-30 21:01:23.200605614 +0000 UTC m=+10.721193738"
Oct 30 21:01:23.319786 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.319741    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:23.854523 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.854500    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:23.963338 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:23.963290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:23.963414 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:23.963373    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:24.138732 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:24.138705    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="b29687b67f36e99cf1776d39e87d382e979f957918b6dabb78d9fa77a9c4edd0" exitCode=0
Oct 30 21:01:24.138827 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:24.138796    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"b29687b67f36e99cf1776d39e87d382e979f957918b6dabb78d9fa77a9c4edd0"}
Oct 30 21:01:24.963663 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:24.963620    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:24.963663 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:24.963667    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:24.964394 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:24.963754    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:24.964394 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:24.963865    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:25.963139 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:25.962977    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:25.963266 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:25.963241    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:26.145275 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:26.145248    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="8e99b664fbc430536985dfe6604b43f16fb86e7e566302afbdffc998a5b43f5c" exitCode=0
Oct 30 21:01:26.145589 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:26.145292    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"8e99b664fbc430536985dfe6604b43f16fb86e7e566302afbdffc998a5b43f5c"}
Oct 30 21:01:26.952372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:26.952340    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:26.963315 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:26.963292    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:26.963461 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:26.963292    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:26.963461 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:26.963411    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:26.963571 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:26.963500    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:27.149631 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.149610    2579 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="1b56ce1bc45d7d265c2e4c3aae89572a176db167430818019dcdb1f6810352b8" exitCode=0
Oct 30 21:01:27.149930 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.149647    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerDied","Data":"1b56ce1bc45d7d265c2e4c3aae89572a176db167430818019dcdb1f6810352b8"}
Oct 30 21:01:27.240673 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.240645    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.363320 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.363293    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.480334 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.480281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.581579 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.581549    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.696353 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.696322    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:27.963003 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:27.962979    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:27.963103 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:27.963056    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:28.097604 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.097583    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:28.154724 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.154702    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" event={"ID":"6f9a963b-963f-4195-9041-faa91154dc88","Type":"ContainerStarted","Data":"40a74c2f86d1ff58c683c6e7b572cd4ef8517f0ba465a1f659b1adab49866abc"}
Oct 30 21:01:28.177038 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.176999    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vzxd4" podStartSLOduration=6.036880809 podStartE2EDuration="15.176988249s" totalImagesPullingTime="9.14010744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:28.17673549 +0000 UTC m=+15.697323614" watchObservedRunningTime="2026-10-30 21:01:28.176988249 +0000 UTC m=+15.697576374"
Oct 30 21:01:28.201204 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.201189    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6gb6m_c2020321-9a3d-4786-b6cb-f00ad50011be/dns-node-resolver/0.log"
Oct 30 21:01:28.893147 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.893122    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:28.963525 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.963499    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:28.963525 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:28.963510    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:28.963636 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:28.963594    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:28.963722 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:28.963706    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:29.103124 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.103103    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.200680 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.200631    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkbt2_4d19a8f3-e037-4087-8d58-f1e05facef69/node-ca/0.log"
Oct 30 21:01:29.240166 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.240142    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.341259 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.341238    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.487889 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.487845    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.586196 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.586175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.624271 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.624249    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:29.624352 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.624284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:29.624399 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.624362    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:29.624399 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.624375    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:29.624491 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.624402    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:01:45.624389608 +0000 UTC m=+33.144977725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:29.624491 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.624419    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:45.624407083 +0000 UTC m=+33.144995186 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:29.690545 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.690524    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:29.725123 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.725104    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:29.725212 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.725205    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:29.725259 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.725216    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:29.725259 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.725224    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:29.725259 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.725255    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:01:45.725246841 +0000 UTC m=+33.245834958 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:29.962986 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:29.962964    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:29.963067 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:29.963041    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:30.144074 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:30.144044    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:30.905257 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:30.905222    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:30.963575 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:30.963547    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:30.963720 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:30.963688    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:30.964015 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:30.963996    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:30.964103 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:30.964077    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:31.019938 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.019909    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.126897 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.126865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.298885 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.298788    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.495397 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.495365    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.707742 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.707680    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:31.963145 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:31.963075    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:31.963679 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:31.963206    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:32.783665 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:32.783622    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:32.903709 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:32.903679    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:32.964858 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:32.964822    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:32.965236 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:32.964920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:32.965236 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:32.964998    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:32.965335 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:32.965296    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:33.102620 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.102560    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.292224 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.292190    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.394891 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.394865    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.501964 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.501929    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.691367 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.691292    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:33.963681 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:33.963605    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:33.963803 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:33.963733    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:34.891976 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:34.891944    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:34.963149 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:34.963124    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:34.963332 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:34.963304    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:34.963429 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:34.963410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:34.963556 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:34.963539    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:35.098013 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.097976    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.310825 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.310758    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.505927 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.505894    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.690673 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.690640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.791160 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.791126    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:35.962820 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:35.962747    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:35.963287 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:35.962863    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:36.896370 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:36.896338    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:36.962865 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:36.962831    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:36.963302 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:36.962951    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:36.963302 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:36.963007    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:36.963302 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:36.963118    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:37.092280 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.092244    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.298138 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.298062    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.409375 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.409329    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.531193 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.531156    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.695476 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.695430    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:37.963152 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:37.963071    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:37.963586 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:37.963216    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:38.089969 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:38.089941    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:38.963198 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:38.963045    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:38.963625 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:38.963090    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:38.963625 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:38.963292    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:38.963625 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:38.963346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:39.177818 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177753    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"65eed67d33d3950faf1a837d54a6260c14af92c07cda845aec2ece1e87355a3b"}
Oct 30 21:01:39.177818 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177813    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"1762b0ab2abfcb2303478d5b1d7c7c1efbc3a9f4593519ca01615f576ba6f873"}
Oct 30 21:01:39.177951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177842    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"4e42c413d952d84d304308708f62d0bdfcc390848a0f46f7b062e9a332acec2d"}
Oct 30 21:01:39.177951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177871    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"80ed241824d580037c78c327b7381f0088a2f7206521685cdd9af0ef8210ae51"}
Oct 30 21:01:39.177951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177897    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"620be13accd78b54f3c82782f15db2b8f55cc9dceb63363e350106897e153b94"}
Oct 30 21:01:39.177951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.177924    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"4695412ce3cca1454eb6ca265a16ef4615c384b64ca2213b6e4b222542e380f4"}
Oct 30 21:01:39.692871 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.692842    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:39.800365 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.800336    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:39.901039 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.901017    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:39.962938 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:39.962896    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:39.963009 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:39.962978    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:40.001848 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.001814    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.114755 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.114720    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.221882 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.221826    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.376191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.376174    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.603659 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.603606    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.713089 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.713069    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:40.963144 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.963125    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:40.963144 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:40.963134    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:40.963282 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:40.963218    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:40.963344 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:40.963327    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:41.185358 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:41.185321    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"63f779c275b6a717e53659278af6275e0d60cb546d6e4e80a34cec4921b45f45"}
Oct 30 21:01:41.963358 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:41.963333    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:41.963504 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:41.963414    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:42.963613 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:42.963583    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:42.963977 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:42.963673    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:42.963977 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:42.963782    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:42.963977 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:42.963899    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:43.963334 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:43.963159    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:43.963536 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:43.963402    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:44.194339 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.194310    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" event={"ID":"a209dc6c-34cd-4fd3-90f2-2591f28988a8","Type":"ContainerStarted","Data":"94b5bf52f7345816d62aed39c6552cf2ed0b6f1352d1c3c3603ed0e3fa148010"}
Oct 30 21:01:44.197917 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.197887    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:44.198058 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.198045    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:44.198121 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.198098    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:44.213400 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.213320    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:44.213618 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.213509    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:01:44.226643 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.226604    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr" podStartSLOduration=6.682930962 podStartE2EDuration="31.226593809s" totalImagesPullingTime="24.543662847s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:44.226119133 +0000 UTC m=+31.746707257" watchObservedRunningTime="2026-10-30 21:01:44.226593809 +0000 UTC m=+31.747181933"
Oct 30 21:01:44.963018 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.962989    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:44.963018 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:44.963020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:44.963205 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:44.963091    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:44.963205 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:44.963168    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:45.530509 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.530472    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:45.646072 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.646048    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:45.646213 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.646096    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:45.646213 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.646179    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:45.646213 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.646188    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:45.646308 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.646223    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret podName:8e34a640-bd21-49d6-a1de-487db5d06085 nodeName:}" failed. No retries permitted until 2026-10-30 21:02:17.646210923 +0000 UTC m=+65.166799038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret") pod "global-pull-secret-syncer-26bcq" (UID: "8e34a640-bd21-49d6-a1de-487db5d06085") : object "kube-system"/"original-pull-secret" not registered
Oct 30 21:01:45.646308 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.646236    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs podName:81a6d809-1b40-4272-9366-466dec6031cf nodeName:}" failed. No retries permitted until 2026-10-30 21:02:17.646229892 +0000 UTC m=+65.166817995 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs") pod "network-metrics-daemon-ctlvw" (UID: "81a6d809-1b40-4272-9366-466dec6031cf") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 21:01:45.712807 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.712774    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:45.747161 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.747131    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:45.747287 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.747251    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 21:01:45.747287 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.747266    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 21:01:45.747287 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.747275    2579 projected.go:196] Error preparing data for projected volume kube-api-access-bpk5q for pod openshift-network-diagnostics/network-check-target-nkrzz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:45.747393 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.747319    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q podName:503db7ff-86df-4655-99e4-27d6db14c835 nodeName:}" failed. No retries permitted until 2026-10-30 21:02:17.747307396 +0000 UTC m=+65.267895499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bpk5q" (UniqueName: "kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q") pod "network-check-target-nkrzz" (UID: "503db7ff-86df-4655-99e4-27d6db14c835") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 21:01:45.962909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:45.962884    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:45.963091 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:45.962985    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:46.072859 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.072777    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26bcq"]
Oct 30 21:01:46.073003 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.072898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:46.073061 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:46.073005    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:46.074845 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.074824    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkrzz"]
Oct 30 21:01:46.075340 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.075324    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ctlvw"]
Oct 30 21:01:46.075411 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.075401    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:46.075514 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:46.075498    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:46.198097 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:46.198079    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:46.198168 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:46.198154    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:47.562722 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:47.562657    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6gb6m_c2020321-9a3d-4786-b6cb-f00ad50011be/dns-node-resolver/0.log"
Oct 30 21:01:47.963793 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:47.963767    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:47.963945 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:47.963800    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:47.963945 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:47.963867    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:47.963945 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:47.963938    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:47.964105 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:47.964036    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:47.964105 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:47.964085    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:48.213645 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:48.213575    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:48.762243 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:48.762222    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkbt2_4d19a8f3-e037-4087-8d58-f1e05facef69/node-ca/0.log"
Oct 30 21:01:49.396890 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:49.396856    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.500688 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:49.500640    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:49.963114 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:49.963088    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:49.963499 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:49.963177    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:49.963499 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:49.963209    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26bcq" podUID="8e34a640-bd21-49d6-a1de-487db5d06085"
Oct 30 21:01:49.963499 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:49.963280    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ctlvw" podUID="81a6d809-1b40-4272-9366-466dec6031cf"
Oct 30 21:01:49.963499 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:49.963305    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:49.963499 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:01:49.963398    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkrzz" podUID="503db7ff-86df-4655-99e4-27d6db14c835"
Oct 30 21:01:51.002567 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.002531    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:51.110641 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.110602    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:51.893148 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.893123    2579 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeReady"
Oct 30 21:01:51.893250 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.893214    2579 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 21:01:51.944645 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.944623    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jmdvh"]
Oct 30 21:01:51.947765 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.947748    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hlskz"]
Oct 30 21:01:51.947881 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.947865    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:51.950707 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.950679    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:51.951279 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.951261    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:01:51.952933 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.952917    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cp8br\""
Oct 30 21:01:51.953083 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.953070    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:01:51.957146 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.956977    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jmdvh"]
Oct 30 21:01:51.960069 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.960047    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hlskz"]
Oct 30 21:01:51.962798 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.962768    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:01:51.962898 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.962871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:01:51.962958 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.962901    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:01:51.968822 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.968804    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v6xnw"]
Oct 30 21:01:51.971591 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.971576    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:51.981156 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:51.981113    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6xnw"]
Oct 30 21:01:52.005644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.005620    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.051873 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.051854    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:01:52.051961 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.051943    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:01:52.052029 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.051998    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:01:52.052029 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.052005    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:01:52.052029 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.052000    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:01:52.052124 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.052039    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:01:52.052124 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.052100    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:01:52.053364 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.053352    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:01:52.053421 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.053411    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:01:52.053495 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.053484    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-s7wtw\""
Oct 30 21:01:52.054108 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054096    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6m5dr\""
Oct 30 21:01:52.054141 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054129    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:01:52.054400 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054389    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6bj8t\""
Oct 30 21:01:52.054444 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054404    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:01:52.054444 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054416    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-qd7pl\""
Oct 30 21:01:52.054505 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.054444    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xpqzq\""
Oct 30 21:01:52.090811 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.090784    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.090900 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.090814    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.090900 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.090844    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.090900 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.090860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.090990 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.090918    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.091027 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091014    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.091068 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091057    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99jdp\" (UniqueName: \"kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.091126 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091113    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.091158 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091135    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58sqv\" (UniqueName: \"kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.091191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091182    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.091222 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.091214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z67ph\" (UniqueName: \"kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.116744 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.116721    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.191920 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.191899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99jdp\" (UniqueName: \"kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.191999 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.191926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.191999 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.191951    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58sqv\" (UniqueName: \"kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.191999 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.191969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.192157 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192064    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z67ph\" (UniqueName: \"kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.192157 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192157 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192152    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192298 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192179    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.192298 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.192298 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192298 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192267    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192298 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.192672 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192652    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192769 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192674    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.192894 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192873    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.192953 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.192876    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.195947 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.195925    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.196039 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.195986    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.196094 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.196047    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.201531 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.201507    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z67ph\" (UniqueName: \"kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.201707 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.201692    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99jdp\" (UniqueName: \"kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.202399 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.202381    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58sqv\" (UniqueName: \"kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.216650 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.216627    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.263228 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.263181    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:52.267103 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.267086    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:01:52.296448 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.296392    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:01:52.345892 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.345743    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.453137 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.453112    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:52.550909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.550853    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hlskz"]
Oct 30 21:01:52.555749 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.555725    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jmdvh"]
Oct 30 21:01:52.572978 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:52.572956    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v6xnw"]
Oct 30 21:01:52.576279 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:52.576251    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0be2b4_c6f4_4366_9d4b_a9b42f23c19c.slice/crio-f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2 WatchSource:0}: Error finding container f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2: Status 404 returned error can't find the container with id f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2
Oct 30 21:01:53.215734 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:53.215678    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6xnw" event={"ID":"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c","Type":"ContainerStarted","Data":"5bcb97c2bc2228623201fdd47dbd46879cf2873f65fb52ec2d5db580fcd802d3"}
Oct 30 21:01:53.216541 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:53.215772    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6xnw" event={"ID":"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c","Type":"ContainerStarted","Data":"f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2"}
Oct 30 21:01:53.218004 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:53.217971    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hlskz" event={"ID":"1699b579-56d9-40a2-a5c5-8253c97fd0d8","Type":"ContainerStarted","Data":"6f3503acdfc475b7d0f58d7adaaf0c8a3434b2b9a610d947231e8442c70f262e"}
Oct 30 21:01:53.219455 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:53.219411    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmdvh" event={"ID":"7ad08418-9a79-4912-831e-d49f5802f695","Type":"ContainerStarted","Data":"6235887f50a87b6e308b9a5350e330860034780a87777c44b9143d80c4df6af8"}
Oct 30 21:01:54.068881 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.068635    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.185471 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.185367    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.232165 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.232084    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6xnw" event={"ID":"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c","Type":"ContainerStarted","Data":"6b6947bbe09e0412f6e60e19da43757ac2dffc866c8469c13311573c14cdd312"}
Oct 30 21:01:54.329314 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.329232    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.445288 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.445249    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.519099 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.519060    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"]
Oct 30 21:01:54.525676 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.524205    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.533391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.533366    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"]
Oct 30 21:01:54.541201 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.538886    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:01:54.541201 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.539975    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-srnnf"]
Oct 30 21:01:54.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.542807    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:01:54.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.542911    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:01:54.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.543002    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:01:54.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.543063    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:01:54.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.543302    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9b2tx\""
Oct 30 21:01:54.545642 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.545622    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"]
Oct 30 21:01:54.550724 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.550703    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.553735 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.551502    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.561681 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.561659    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"]
Oct 30 21:01:54.578135 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.577727    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:01:54.578135 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.577784    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4f5fg\""
Oct 30 21:01:54.578135 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.577925    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:01:54.578135 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.578018    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gxjw5\""
Oct 30 21:01:54.578135 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.578081    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:01:54.578587 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.578559    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:01:54.592264 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.592209    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.612726 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612705    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.612857 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.612857 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612802    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.612857 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612835    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w26g\" (UniqueName: \"kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.613030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612870    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612897    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612927    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.612969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.613030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613002    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613033    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613074    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613117    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn2qz\" (UniqueName: \"kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613138    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613157    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613172    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.613256 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613230    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.613605 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.613261    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxlxg\" (UniqueName: \"kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.638143 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.637900    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:01:54.638143 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.638116    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:01:54.703508 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.703478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.713773 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.713894 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713789    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.713894 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713817    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn2qz\" (UniqueName: \"kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.713894 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713851    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.713894 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713878    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713904    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713932    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.713975    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714008    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxlxg\" (UniqueName: \"kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714101 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714074    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714113    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714151    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w26g\" (UniqueName: \"kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714186    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714271    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.714383 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714352    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.714753 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.714753 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714477    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714753 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714614    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.714753 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.714632    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.715817 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.715417    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.715912 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.715428    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.715983 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.715665    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.715983 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.715705    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.716330 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.716309    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.716583 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.716556    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.725072 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.725048    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxlxg\" (UniqueName: \"kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.725314 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.725291    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w26g\" (UniqueName: \"kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.726513 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.726491    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn2qz\" (UniqueName: \"kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.729880 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.729860    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.729989 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.729860    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.729989 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.729909    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.729989 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.729979    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.730131 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.730035    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.731878 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.731854    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.861009 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.860973    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:01:54.861930 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.861903    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:54.931861 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.928887    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:01:54.939914 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.939893    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:01:54.945604 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:54.945580    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ce6b46c_9dba_4f64_bb0e_50287d8b5d8f.slice/crio-8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743 WatchSource:0}: Error finding container 8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743: Status 404 returned error can't find the container with id 8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743
Oct 30 21:01:54.974073 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:54.974018    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:55.078476 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.078424    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:55.150092 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.149954    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"]
Oct 30 21:01:55.153478 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:55.153421    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068a733e_f3f0_4aec_9db1_a78b9e81c332.slice/crio-cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc WatchSource:0}: Error finding container cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc: Status 404 returned error can't find the container with id cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc
Oct 30 21:01:55.227481 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.227458    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"]
Oct 30 21:01:55.237768 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.237740    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v6xnw" event={"ID":"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c","Type":"ContainerStarted","Data":"d10806a5f527a42ff15625820d3aaa662cb8092fc55dd00ffb8483755855ffc0"}
Oct 30 21:01:55.239427 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.239192    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n" event={"ID":"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7","Type":"ContainerStarted","Data":"239e3a9301d09602cfebfe17039b2020c433914d534a211ad149492a53933982"}
Oct 30 21:01:55.240951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.240926    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srnnf" event={"ID":"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f","Type":"ContainerStarted","Data":"8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743"}
Oct 30 21:01:55.242770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.242741    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s" event={"ID":"068a733e-f3f0-4aec-9db1-a78b9e81c332","Type":"ContainerStarted","Data":"3af0094c8c6003ef272d52f646561163326cabf6d74f82d05ad7c84f454f4961"}
Oct 30 21:01:55.242866 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.242797    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s" event={"ID":"068a733e-f3f0-4aec-9db1-a78b9e81c332","Type":"ContainerStarted","Data":"cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc"}
Oct 30 21:01:55.244617 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.244592    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmdvh" event={"ID":"7ad08418-9a79-4912-831e-d49f5802f695","Type":"ContainerStarted","Data":"19857f9793185e8012db94f63d470eb001153a70dd76691a9c5058594d2581df"}
Oct 30 21:01:55.244714 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.244639    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jmdvh" event={"ID":"7ad08418-9a79-4912-831e-d49f5802f695","Type":"ContainerStarted","Data":"62302cd319124d0015b2d037b374664cff689179cf6f2e15fc8e73b6fa5642d7"}
Oct 30 21:01:55.244773 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.244753    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:01:55.257194 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.257154    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v6xnw" podStartSLOduration=1.871710856 podStartE2EDuration="4.257143528s" totalImagesPullingTime="2.385432672s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:55.256654828 +0000 UTC m=+42.777242953" watchObservedRunningTime="2026-10-30 21:01:55.257143528 +0000 UTC m=+42.777731655"
Oct 30 21:01:55.551218 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.551184    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:55.610084 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.610017    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-jmdvh" podStartSLOduration=2.676118286 podStartE2EDuration="4.609996271s" totalImagesPullingTime="1.933877985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:55.276585907 +0000 UTC m=+42.797174032" watchObservedRunningTime="2026-10-30 21:01:55.609996271 +0000 UTC m=+43.130584397"
Oct 30 21:01:55.610392 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.610365    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:01:55.614231 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.614200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.623161 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.623138    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:01:55.635535 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.635430    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:01:55.687081 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687052    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:01:55.687081 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687052    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:01:55.687314 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687297    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:01:55.687393 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687350    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:01:55.687393 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687354    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lgbjz\""
Oct 30 21:01:55.687393 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687348    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:01:55.687512 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687429    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:01:55.687512 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687457    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:01:55.687512 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.687428    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:01:55.722962 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.722901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.722962 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.722934    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.722962 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.722953    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723163 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723040    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723163 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723090    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723249 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723249 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723199    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723249 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723249 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723240    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723395 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723281    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723395 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723304    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723395 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723335    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.723395 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.723360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.772841 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.772761    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:55.824069 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824035    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824069 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824082    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824143    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824171    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824235    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824426 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824298    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824426 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824426 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824357    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824426 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824392    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824453    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.824644 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.824476    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.825679 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.825037    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.825679 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.825351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.826489 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.826409    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.832909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.831501    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.832909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.832055    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.832909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.832793    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.833189 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.833163    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.834062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.833492    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.834062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.833683    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.834062 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.834019    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.834239 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.834116    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.834998 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.834976    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.836406 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.836360    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.943197 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.943164    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:01:55.972085 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:55.972056    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:56.240802 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.240652    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:01:56.246510 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:56.246479    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf666bd4a_175a_42dd_adb8_11aaf725c174.slice/crio-a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb WatchSource:0}: Error finding container a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb: Status 404 returned error can't find the container with id a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb
Oct 30 21:01:56.250279 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.250186    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s" event={"ID":"068a733e-f3f0-4aec-9db1-a78b9e81c332","Type":"ContainerStarted","Data":"cd85cb6633426af26e9e4ec13abbb0c88fa35c3b44729b0a0c5f20c4a700f86a"}
Oct 30 21:01:56.251925 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.251898    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hlskz" event={"ID":"1699b579-56d9-40a2-a5c5-8253c97fd0d8","Type":"ContainerStarted","Data":"9cde2c0574e34560fb20eb40b7288f6dd3f8c85d42f6c72eea5129bec5435675"}
Oct 30 21:01:56.594763 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.594720    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hlskz" podStartSLOduration=2.702224469 podStartE2EDuration="5.594705602s" totalImagesPullingTime="2.892481133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:56.268060418 +0000 UTC m=+43.788648542" watchObservedRunningTime="2026-10-30 21:01:56.594705602 +0000 UTC m=+44.115293727"
Oct 30 21:01:56.595216 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.595199    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-756469fbcb-g9zw7"]
Oct 30 21:01:56.601974 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.601953    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.610180 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.610159    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-756469fbcb-g9zw7"]
Oct 30 21:01:56.698693 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.698668    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:01:56.698782 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.698719    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:01:56.699195 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.699177    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:01:56.699372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.699355    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1t1guua5phs70\""
Oct 30 21:01:56.699421 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.699407    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:01:56.699505 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.699487    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2j8dz\""
Oct 30 21:01:56.699542 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.699498    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8fh2\" (UniqueName: \"kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735370    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735548    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735620    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735667    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.736467 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.735709    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836642    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836669    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836714    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836734    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8fh2\" (UniqueName: \"kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836751    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.836856    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.837823 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.837190    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.839301 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.838962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.841737 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.841715    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.841834 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.841797    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.842007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.841984    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.842249 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.842227    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.842848 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.842824    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.844104 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.844079    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.845877 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.845852    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8fh2\" (UniqueName: \"kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:56.932075 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:56.932047    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:01:57.222080 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.222049    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-756469fbcb-g9zw7"]
Oct 30 21:01:57.225673 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:57.225643    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78874e1_9201_4cfa_871f_1af75327111b.slice/crio-ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6 WatchSource:0}: Error finding container ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6: Status 404 returned error can't find the container with id ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6
Oct 30 21:01:57.258223 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.258194    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n" event={"ID":"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7","Type":"ContainerStarted","Data":"c777f7ff48d627e027dbc9b14a09f7bed4bef52792627d9adf840cb12d09d833"}
Oct 30 21:01:57.258829 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.258249    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n" event={"ID":"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7","Type":"ContainerStarted","Data":"6db8940a2a6f8b328435b14ccf147c853860713868b804c4e4a04a073456567b"}
Oct 30 21:01:57.258829 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.258271    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n" event={"ID":"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7","Type":"ContainerStarted","Data":"2b408ec2ca9cd2e26c9fea5ccb79181ec2c58d9b7ec82983f1bdd6e94d7c60ba"}
Oct 30 21:01:57.259640 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.259604    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb"}
Oct 30 21:01:57.261341 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.261315    2579 generic.go:356] "Generic (PLEG): container finished" podID="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" containerID="f8728e7d3cc8210218f34a4628d7bfa5b0a213426c9c0666de5f506c9dce6515" exitCode=0
Oct 30 21:01:57.261475 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.261382    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srnnf" event={"ID":"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f","Type":"ContainerDied","Data":"f8728e7d3cc8210218f34a4628d7bfa5b0a213426c9c0666de5f506c9dce6515"}
Oct 30 21:01:57.263973 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.263949    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s" event={"ID":"068a733e-f3f0-4aec-9db1-a78b9e81c332","Type":"ContainerStarted","Data":"5cce50e19849d2b2f778cda7fc151fc25017ce27e5e57bc5e69a63a7f7954bad"}
Oct 30 21:01:57.265311 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.265290    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6"}
Oct 30 21:01:57.299311 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.299268    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n" podStartSLOduration=1.963150105 podStartE2EDuration="3.299256078s" totalImagesPullingTime="1.336105973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:57.277569905 +0000 UTC m=+44.798158030" watchObservedRunningTime="2026-10-30 21:01:57.299256078 +0000 UTC m=+44.819844203"
Oct 30 21:01:57.321230 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.321187    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s" podStartSLOduration=2.144687588 podStartE2EDuration="3.321173211s" totalImagesPullingTime="1.176485623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:57.319908565 +0000 UTC m=+44.840496691" watchObservedRunningTime="2026-10-30 21:01:57.321173211 +0000 UTC m=+44.841761330"
Oct 30 21:01:57.563421 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.563344    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:57.669323 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.669281    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:57.784203 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.784168    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:57.972696 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:57.972629    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:58.269309 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.269231    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:58.272502 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.272474    2579 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a" exitCode=0
Oct 30 21:01:58.272637 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.272507    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"}
Oct 30 21:01:58.277492 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.277466    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srnnf" event={"ID":"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f","Type":"ContainerStarted","Data":"809614bd222d603703f7221b7269924ef12619e821418fb8d84f82d0c57bc1ad"}
Oct 30 21:01:58.277635 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.277516    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-srnnf" event={"ID":"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f","Type":"ContainerStarted","Data":"9269f8625650cb941446b012c3eaf181b08bb2c21618b36e8d4bd5627554ffb3"}
Oct 30 21:01:58.318585 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.318526    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-srnnf" podStartSLOduration=2.9672987859999997 podStartE2EDuration="4.318507307s" totalImagesPullingTime="1.265011987s" totalInitContainerRuntime="86.196534ms" isStatefulPod=true podCreationTimestamp="2026-10-30 21:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:01:58.31757816 +0000 UTC m=+45.838166287" watchObservedRunningTime="2026-10-30 21:01:58.318507307 +0000 UTC m=+45.839095436"
Oct 30 21:01:58.524372 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.524282    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:58.628981 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.628942    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:58.931268 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.930181    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-8b68d8d9-5tfbm"]
Oct 30 21:01:58.933393 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.933371    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:58.939185 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.939166    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:01:58.943554 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.943484    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8b68d8d9-5tfbm"]
Oct 30 21:01:58.967872 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.967823    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:01:58.967872 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.967826    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:01:58.968080 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.967909    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3ji7goluo3bfd\""
Oct 30 21:01:58.968080 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:58.967960    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dsnfd\""
Oct 30 21:01:59.037863 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.037832    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:01:59.060290 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060255    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42vnz\" (UniqueName: \"kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060517 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060347    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060605 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060671 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060627    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060730 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060672    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060783 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060741    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.060900 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.060879    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162017 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.161985    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42vnz\" (UniqueName: \"kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162184 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162156    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162250 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162250 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162221    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162360 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162360 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162360 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162342    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.162893 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.162865    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.163146 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.163117    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.163865 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.163828    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.165709 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.165685    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.165800 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.165710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.165800 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.165726    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.171654 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.171626    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42vnz\" (UniqueName: \"kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.250463 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.249035    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:01:59.284979 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.284227    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"bbb8ef38a64aafaf89dbc42c4116860e9e2204c70415aa79517db039d2230e4d"}
Oct 30 21:01:59.323012 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.322589    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"]
Oct 30 21:01:59.330193 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.325990    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.333735 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.333386    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zxwnt\""
Oct 30 21:01:59.333860 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.333774    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:01:59.336598 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.336571    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"]
Oct 30 21:01:59.359649 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.359577    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:59.465577 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.465490    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w665x\" (UniqueName: \"kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.465577 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.465559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.541169 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.541137    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-8b68d8d9-5tfbm"]
Oct 30 21:01:59.544007 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.543983    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:01:59.545195 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:59.545166    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3abab98_e4d8_4ebd_b4a3_81d91079ab72.slice/crio-24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0 WatchSource:0}: Error finding container 24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0: Status 404 returned error can't find the container with id 24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0
Oct 30 21:01:59.566741 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.566715    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w665x\" (UniqueName: \"kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.566853 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.566771    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.569420 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.569401    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.574857 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.574837    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w665x\" (UniqueName: \"kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.646893 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.646862    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:01:59.923893 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:01:59.923871    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"]
Oct 30 21:01:59.928261 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:01:59.928233    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa94833_b283_4190_b623_f36b7c9edf2f.slice/crio-02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5 WatchSource:0}: Error finding container 02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5: Status 404 returned error can't find the container with id 02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5
Oct 30 21:02:00.292145 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.292071    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"a047418519bc39ad6677fd248b82e7d68f1b0327dcfe883bd0697ae361a9167d"}
Oct 30 21:02:00.292145 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.292141    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"e3248c8ad3041e4d43b151dc33ff93b15653c33ce2de39649c2da13b4e664b9b"}
Oct 30 21:02:00.295353 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.295328    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"}
Oct 30 21:02:00.295514 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.295392    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"}
Oct 30 21:02:00.295514 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.295448    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"}
Oct 30 21:02:00.295514 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.295485    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"}
Oct 30 21:02:00.295657 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.295519    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"}
Oct 30 21:02:00.296779 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.296751    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx" event={"ID":"4fa94833-b283-4190-b623-f36b7c9edf2f","Type":"ContainerStarted","Data":"02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5"}
Oct 30 21:02:00.298070 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.298021    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm" event={"ID":"d3abab98-e4d8-4ebd-b4a3-81d91079ab72","Type":"ContainerStarted","Data":"24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0"}
Oct 30 21:02:00.816981 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.816947    2579 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:00.821097 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.821073    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826362    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826428    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826429    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826464    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7dnlj6d6sts0u\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826525    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826537    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826547    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826579    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:02:00.826809 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.826653    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:02:00.827456 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.827424    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:02:00.827646 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.827606    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9ljfc\""
Oct 30 21:02:00.827914 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.827895    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:02:00.828000 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.827957    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:02:00.828186 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.828168    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:02:00.836580 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.836349    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:02:00.836820 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.836687    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:02:00.840885 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.839405    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:00.866286 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.866258    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:00.977155 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977127    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977323 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977323 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977189    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977410 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977410 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977410 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977551 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977583 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977552    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977583 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977567    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977653 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977653 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977739 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977652    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977800 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977837 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977807    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977837 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977825    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977901 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977951 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977900    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.977987 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.978016 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.977984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:00.978050 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:00.978030    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079126 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079043    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079126 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079126 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079172    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079192    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079214    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079255    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079276    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079301    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079391 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079336    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079429    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079532    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079588    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079615    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079658    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079684    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.079760 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.079718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.080672 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.080644    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.081002 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.080981    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.083525 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.083493    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.083721 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.083700    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.084191 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.084163    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.084993 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.084966    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.085959 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.085932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.086235 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.086213    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.086347 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.086323    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.086545 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.086523    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.086978 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.086956    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.087351 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.087323    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.087605 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.087561    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.087906 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.087803    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.088049 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.088028    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.088111 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.088080    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.089261 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.089231    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.090953 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.090028    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.090953 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.090225    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.095641 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.095612    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.149501 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.149471    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:01.305909 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.305858    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerStarted","Data":"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"}
Oct 30 21:02:01.309154 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.309126    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"c633054e5dfc93cdbee8696980b923f4f6a17b6e81cd5b2d68788728bae7808d"}
Oct 30 21:02:01.309294 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.309184    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"d37f4fdb4c8a2da0ed37109b20f015dc94e556b81fbbeeaa3a52c003ecf0afcc"}
Oct 30 21:02:01.309294 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.309209    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" event={"ID":"b78874e1-9201-4cfa-871f-1af75327111b","Type":"ContainerStarted","Data":"3a9dec83d0ea31c48014e8ddd6e5bd8ec3166674384c96550e0502e8011fda27"}
Oct 30 21:02:01.309572 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.309326    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:01.334881 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.334807    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.316880475 podStartE2EDuration="6.334791387s" totalImagesPullingTime="3.003886642s" totalInitContainerRuntime="14.02427ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:01.333162769 +0000 UTC m=+48.853750894" watchObservedRunningTime="2026-10-30 21:02:01.334791387 +0000 UTC m=+48.855379493"
Oct 30 21:02:01.445415 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.445350    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7" podStartSLOduration=2.572639389 podStartE2EDuration="5.445332847s" totalImagesPullingTime="2.872693458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:01.358223974 +0000 UTC m=+48.878812101" watchObservedRunningTime="2026-10-30 21:02:01.445332847 +0000 UTC m=+48.965920974"
Oct 30 21:02:01.446394 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.446352    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:01.448986 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:02:01.448957    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4acaeddc_b0dd_451a_8cbe_353b0a0823af.slice/crio-640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f WatchSource:0}: Error finding container 640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f: Status 404 returned error can't find the container with id 640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f
Oct 30 21:02:01.449090 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.449012    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:01.549179 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:01.549139    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:02.313994 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.313953    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx" event={"ID":"4fa94833-b283-4190-b623-f36b7c9edf2f","Type":"ContainerStarted","Data":"8c5e9320b1c2caea5af75ec9288657f76dafcf18dbb4502ecdf89f112ba369b3"}
Oct 30 21:02:02.314468 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.314196    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:02.315802 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.315776    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm" event={"ID":"d3abab98-e4d8-4ebd-b4a3-81d91079ab72","Type":"ContainerStarted","Data":"850468f61a27d60f8ea946e77275500ac71ae7eefcd1044b2d056811e61082da"}
Oct 30 21:02:02.317491 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.317469    2579 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e" exitCode=0
Oct 30 21:02:02.317611 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.317556    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"}
Oct 30 21:02:02.317679 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.317616    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f"}
Oct 30 21:02:02.319479 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.319464    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:02.329529 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.329494    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx" podStartSLOduration=1.7691854299999998 podStartE2EDuration="3.329484156s" totalImagesPullingTime="1.560298726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:02.328604075 +0000 UTC m=+49.849192202" watchObservedRunningTime="2026-10-30 21:02:02.329484156 +0000 UTC m=+49.850072276"
Oct 30 21:02:02.365538 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:02.365501    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm" podStartSLOduration=2.591161584 podStartE2EDuration="4.365486256s" totalImagesPullingTime="1.774324672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:02.364355864 +0000 UTC m=+49.884943989" watchObservedRunningTime="2026-10-30 21:02:02.365486256 +0000 UTC m=+49.886074381"
Oct 30 21:02:04.425688 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:04.425654    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:04.522181 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:04.522145    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:05.255030 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:05.255007    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:05.333345 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:05.333315    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"}
Oct 30 21:02:06.340240 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.340211    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"}
Oct 30 21:02:06.340770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.340295    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"}
Oct 30 21:02:06.340770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.340327    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"}
Oct 30 21:02:06.340770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.340350    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"}
Oct 30 21:02:06.340770 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.340375    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerStarted","Data":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"}
Oct 30 21:02:06.369879 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:06.369832    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.4234343640000002 podStartE2EDuration="6.369812698s" totalImagesPullingTime="2.930244201s" totalInitContainerRuntime="16.134133ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:06.368323176 +0000 UTC m=+53.888911309" watchObservedRunningTime="2026-10-30 21:02:06.369812698 +0000 UTC m=+53.890400824"
Oct 30 21:02:07.223746 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:07.223713    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:07.325929 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:07.325896    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:07.344978 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:07.344951    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:07.469266 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:07.469233    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:09.103278 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:09.103247    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:09.216511 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:09.216478    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:10.007672 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:10.007631    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:10.113077 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:10.113045    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:11.150521 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:11.150490    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:13.154209 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:13.154175    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:13.255575 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:13.255547    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.209502 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:16.209474    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:16.745084 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:16.745053    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:16.872954 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:16.872918    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:17.745330 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.744958    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:17.745330 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.745040    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:17.748120 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.748099    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:02:17.749130 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.749101    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:02:17.758610 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.758587    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:17.758733 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.758634    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:17.779330 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.779311    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6m5dr\""
Oct 30 21:02:17.783996 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.783982    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6bj8t\""
Oct 30 21:02:17.787090 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.787076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:17.792005 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.791983    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:17.851683 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.851176    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:17.856468 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.855535    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:02:17.864522 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.864100    2579 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:02:17.876714 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:17.876648    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:18.077130 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.077109    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ctlvw"]
Oct 30 21:02:18.080252 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:02:18.080223    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81a6d809_1b40_4272_9366_466dec6031cf.slice/crio-b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955 WatchSource:0}: Error finding container b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955: Status 404 returned error can't find the container with id b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955
Oct 30 21:02:18.080549 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.080406    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26bcq"]
Oct 30 21:02:18.082906 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:02:18.082880    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e34a640_bd21_49d6_a1de_487db5d06085.slice/crio-acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0 WatchSource:0}: Error finding container acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0: Status 404 returned error can't find the container with id acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0
Oct 30 21:02:18.090064 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.090045    2579 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-qd7pl\""
Oct 30 21:02:18.097803 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.097779    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:18.369725 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.369704    2579 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkrzz"]
Oct 30 21:02:18.371691 ip-10-0-139-2 kubenswrapper[2579]: W1030 21:02:18.371670    2579 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503db7ff_86df_4655_99e4_27d6db14c835.slice/crio-a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2 WatchSource:0}: Error finding container a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2: Status 404 returned error can't find the container with id a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2
Oct 30 21:02:18.385636 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.385612    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkrzz" event={"ID":"503db7ff-86df-4655-99e4-27d6db14c835","Type":"ContainerStarted","Data":"a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2"}
Oct 30 21:02:18.386697 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.386677    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-26bcq" event={"ID":"8e34a640-bd21-49d6-a1de-487db5d06085","Type":"ContainerStarted","Data":"acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0"}
Oct 30 21:02:18.387699 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:18.387682    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctlvw" event={"ID":"81a6d809-1b40-4272-9366-466dec6031cf","Type":"ContainerStarted","Data":"b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955"}
Oct 30 21:02:19.249149 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:19.249128    2579 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:19.249465 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:19.249178    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:19.394004 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:19.393972    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctlvw" event={"ID":"81a6d809-1b40-4272-9366-466dec6031cf","Type":"ContainerStarted","Data":"b380b4667d9996a6e52521f9df320793d9be3d3e8aeee35e71e2559542a51c10"}
Oct 30 21:02:19.394117 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:19.394030    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ctlvw" event={"ID":"81a6d809-1b40-4272-9366-466dec6031cf","Type":"ContainerStarted","Data":"597b6ecd297c5a5e834c0bdf163c8a0b8287f8fd15eb937fb4b87c9319c9aaac"}
Oct 30 21:02:19.411956 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:19.411895    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ctlvw" podStartSLOduration=65.326898482 podStartE2EDuration="1m6.411877148s" totalImagesPullingTime="1.084978666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:19.410534435 +0000 UTC m=+66.931122560" watchObservedRunningTime="2026-10-30 21:02:19.411877148 +0000 UTC m=+66.932465308"
Oct 30 21:02:22.329721 ip-10-0-139-2 kubenswrapper[2579]: E1030 21:02:22.329684    2579 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:02:22.353683 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:22.353653    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:22.414573 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:22.414536    2579 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkrzz" event={"ID":"503db7ff-86df-4655-99e4-27d6db14c835","Type":"ContainerStarted","Data":"c529790f35b764884a832bdcc7d0d2daaad74a9759a5328bceb85265ae40e0b3"}
Oct 30 21:02:22.415361 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:22.415332    2579 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:22.432690 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:22.432631    2579 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nkrzz" podStartSLOduration=66.073046555 podStartE2EDuration="1m9.432615257s" totalImagesPullingTime="3.359568702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:01:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:22.431305615 +0000 UTC m=+69.951893740" watchObservedRunningTime="2026-10-30 21:02:22.432615257 +0000 UTC m=+69.953203383"
Oct 30 21:02:22.457474 ip-10-0-139-2 kubenswrapper[2579]: I1030 21:02:22.457411    2579 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:22.981494 ip-10-0-139-2 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 21:02:22.989015 ip-10-0-139-2 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 21:02:22.989224 ip-10-0-139-2 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 21:02:22.989417 ip-10-0-139-2 systemd[1]: kubelet.service: Consumed 4.393s CPU time.
Oct 30 21:02:23.004235 ip-10-0-139-2 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 21:02:23.073331 ip-10-0-139-2 kubenswrapper[6738]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:02:23.073331 ip-10-0-139-2 kubenswrapper[6738]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 21:02:23.073331 ip-10-0-139-2 kubenswrapper[6738]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:02:23.073331 ip-10-0-139-2 kubenswrapper[6738]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 21:02:23.075339 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075261    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.075339 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075339    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075343    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075346    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075349    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075352    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075355    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075358    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075361    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075364    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075367    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075370    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075372    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075375    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075377    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075380    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075383    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075385    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075388    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075391    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.075411 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075393    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075396    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075399    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075401    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075404    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075407    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075410    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075413    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075415    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075418    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075420    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075423    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075426    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075429    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075446    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075449    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075453    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075457    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075460    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075463    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.075922 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075466    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075469    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075472    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075474    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075478    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075480    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075483    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075485    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075488    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075491    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075493    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075496    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075498    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075501    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075503    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075506    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075509    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075511    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075514    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.076413 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075516    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075521    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075524    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075527    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075530    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075532    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075535    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075537    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075540    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075542    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075545    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075547    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075550    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075553    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075556    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075558    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075561    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075564    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075566    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.076884 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075569    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075572    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075574    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075576    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075579    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075581    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075584    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075587    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075589    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075591    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075594    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075597    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075599    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075601    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075604    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075606    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075609    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075611    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075613    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.077388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075616    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075618    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075620    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075623    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075625    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075628    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075634    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075637    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.075640    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076390    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076396    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076400    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076402    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076405    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076408    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076410    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076413    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076416    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076418    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.077877 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076421    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076423    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076426    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076429    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076450    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076453    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076456    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076458    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076461    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076465    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076469    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076472    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076475    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076478    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076481    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076484    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076487    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076490    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076493    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.078333 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076497    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076500    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076502    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076506    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076508    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076510    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076513    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076515    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076518    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076520    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076522    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076525    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076527    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076530    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076533    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076536    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076538    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076541    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076545    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076547    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.078810 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076550    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076552    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076555    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076557    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076560    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076562    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076565    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076567    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076570    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076572    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076575    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076577    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076580    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076583    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076585    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076587    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076590    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076593    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076596    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.079312 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076598    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076601    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076603    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076606    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076608    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076611    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076614    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076617    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076620    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076624    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076628    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076631    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076633    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076636    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076638    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076641    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076643    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076646    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076648    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.079800 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076652    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076654    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076656    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076659    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076662    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076664    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076667    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076669    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076672    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076674    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076676    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076682    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076685    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076687    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076690    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076692    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076695    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076697    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.076699    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077200    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.080268 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077206    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077209    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077212    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077215    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077217    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077220    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077222    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077225    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077228    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077230    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077233    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077236    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077238    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077241    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077243    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077246    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077248    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077251    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077254    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077256    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.080767 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077259    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077261    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077263    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077267    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077269    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077272    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077274    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077277    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077281    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077284    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077286    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077289    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077291    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077294    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077297    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077300    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077302    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077304    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077312    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.081299 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077314    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077317    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077319    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077322    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077325    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077328    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077330    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077332    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077335    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077338    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077340    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077343    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077345    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077348    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077350    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077353    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077355    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077361    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077363    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.081881 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077367    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077369    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077373    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077376    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077378    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077381    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077385    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077388    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077391    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077394    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077396    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077399    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077402    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077405    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077407    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077410    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077412    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077415    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077417    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077420    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.082661 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077422    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077425    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077427    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077444    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077447    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077451    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077455    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077458    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077460    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077464    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077466    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077469    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077472    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077474    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077478    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077480    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077483    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077485    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077488    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.083335 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077490    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077493    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077495    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077497    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077500    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077504    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077506    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.077509    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077597    6738 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077613    6738 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077622    6738 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077629    6738 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077636    6738 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077641    6738 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077646    6738 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077650    6738 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077653    6738 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077656    6738 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077659    6738 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077662    6738 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077665    6738 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077668    6738 flags.go:64] FLAG: --cgroup-root=""
Oct 30 21:02:23.083875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077671    6738 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077674    6738 flags.go:64] FLAG: --client-ca-file=""
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077677    6738 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077681    6738 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077686    6738 flags.go:64] FLAG: --cluster-domain=""
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077690    6738 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077694    6738 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077696    6738 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077700    6738 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077703    6738 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077707    6738 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077710    6738 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077713    6738 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077716    6738 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077719    6738 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077723    6738 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077726    6738 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077729    6738 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077733    6738 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077736    6738 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077739    6738 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077742    6738 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077746    6738 flags.go:64] FLAG: --enable-server="true"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077751    6738 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077758    6738 flags.go:64] FLAG: --event-burst="100"
Oct 30 21:02:23.084386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077762    6738 flags.go:64] FLAG: --event-qps="50"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077766    6738 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077772    6738 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077775    6738 flags.go:64] FLAG: --eviction-hard=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077779    6738 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077782    6738 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077785    6738 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077788    6738 flags.go:64] FLAG: --eviction-soft=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077791    6738 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077794    6738 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077797    6738 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077801    6738 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077805    6738 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077808    6738 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077811    6738 flags.go:64] FLAG: --feature-gates=""
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077815    6738 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077817    6738 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077820    6738 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077824    6738 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077827    6738 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077830    6738 flags.go:64] FLAG: --help="false"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077833    6738 flags.go:64] FLAG: --hostname-override="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077837    6738 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077840    6738 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 21:02:23.085028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077843    6738 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077846    6738 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077849    6738 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077852    6738 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077855    6738 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077857    6738 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077860    6738 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077863    6738 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077866    6738 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077869    6738 flags.go:64] FLAG: --kube-reserved=""
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077872    6738 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077874    6738 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077877    6738 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077880    6738 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077882    6738 flags.go:64] FLAG: --lock-file=""
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077885    6738 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077889    6738 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077892    6738 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077897    6738 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077900    6738 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077904    6738 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077908    6738 flags.go:64] FLAG: --logging-format="text"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077910    6738 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077914    6738 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 21:02:23.085638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077917    6738 flags.go:64] FLAG: --manifest-url=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077919    6738 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077924    6738 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077926    6738 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077930    6738 flags.go:64] FLAG: --max-pods="110"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077933    6738 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077936    6738 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077939    6738 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077942    6738 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077945    6738 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077948    6738 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077951    6738 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077958    6738 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077961    6738 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077964    6738 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077967    6738 flags.go:64] FLAG: --pod-cidr=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077969    6738 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077972    6738 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077975    6738 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077978    6738 flags.go:64] FLAG: --port="10250"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077981    6738 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077983    6738 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-03a103c138a44637b"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077986    6738 flags.go:64] FLAG: --qos-reserved=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077989    6738 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077992    6738 flags.go:64] FLAG: --register-node="true"
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077995    6738 flags.go:64] FLAG: --register-with-taints=""
Oct 30 21:02:23.086402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.077998    6738 flags.go:64] FLAG: --registry-burst="10"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078001    6738 flags.go:64] FLAG: --registry-qps="5"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078005    6738 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078008    6738 flags.go:64] FLAG: --reserved-memory=""
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078012    6738 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078015    6738 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078018    6738 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078021    6738 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078023    6738 flags.go:64] FLAG: --runonce="false"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078026    6738 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078029    6738 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078032    6738 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078035    6738 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078038    6738 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078041    6738 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078044    6738 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078047    6738 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078050    6738 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078053    6738 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078055    6738 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078058    6738 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078061    6738 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078064    6738 flags.go:64] FLAG: --system-cgroups=""
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078067    6738 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078072    6738 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 21:02:23.087130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078075    6738 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078078    6738 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078082    6738 flags.go:64] FLAG: --tls-min-version=""
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078085    6738 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078088    6738 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078091    6738 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078093    6738 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078096    6738 flags.go:64] FLAG: --v="2"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078105    6738 flags.go:64] FLAG: --version="false"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078111    6738 flags.go:64] FLAG: --vmodule=""
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078114    6738 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078118    6738 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078251    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078255    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078258    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078260    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078263    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078265    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078268    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078270    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078273    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078275    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078278    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.087904 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078280    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078283    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078286    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078288    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078291    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078293    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078296    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078299    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078301    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078304    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078306    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078309    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078311    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078314    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078316    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078319    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078322    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078324    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078328    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078330    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.088547 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078333    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078336    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078339    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078342    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078344    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078347    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078350    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078353    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078355    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078358    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078360    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078363    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078365    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078368    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078370    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078373    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078375    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078380    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078382    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.089289 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078385    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078389    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078392    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078395    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078398    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078400    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078403    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078405    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078408    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078410    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078413    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078416    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078418    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078421    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078423    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078426    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078429    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078444    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078447    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.089821 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078449    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078452    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078454    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078457    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078459    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078462    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078464    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078467    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078469    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078472    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078474    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078478    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078480    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078483    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078485    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078488    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078490    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078492    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078495    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.090560 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078497    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078500    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078502    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078504    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078507    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078509    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078512    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078514    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078517    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078520    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078523    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078525    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078528    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078530    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078533    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078535    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078538    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.091418 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078542    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.078568    6738 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078603    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078607    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078609    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078612    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078615    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078618    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078621    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078623    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078626    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078629    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078631    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078634    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078636    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078639    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.092019 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078641    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078644    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078646    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078649    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078651    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078654    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078656    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078659    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078661    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078665    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078669    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078672    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078675    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078677    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078680    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078683    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078685    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078687    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078690    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.092549 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078692    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078695    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078697    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078699    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078703    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078706    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078708    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078710    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078713    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078715    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078718    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078720    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078723    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078725    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078728    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078731    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078733    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078735    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078738    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.093200 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078740    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078743    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078745    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078747    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078750    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078752    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078755    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078757    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078760    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078762    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078765    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078767    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078769    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078772    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078774    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078776    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078780    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078784    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078786    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078789    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.093689 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078791    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078794    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078797    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078799    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078802    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078806    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078810    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078813    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078816    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078819    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078821    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078824    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078826    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078829    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078831    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078833    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078836    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078838    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078841    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078843    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.094196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078846    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078849    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078851    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078854    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078856    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078859    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078861    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078864    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078867    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078872    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078874    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078877    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078879    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.078883    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.083967    6738 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.083984    6738 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.084003    6738 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.084008    6738 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084064    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.094695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084070    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084074    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084078    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084080    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084083    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084085    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084088    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084091    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084093    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084096    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084098    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084101    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084104    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084108    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084112    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084115    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084119    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084122    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084125    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084128    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.095163 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084131    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084134    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084137    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084139    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084142    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084144    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084147    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084149    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084153    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084156    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084158    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084161    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084163    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084166    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084169    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084171    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084174    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084176    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084179    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084182    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.095741 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084185    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084187    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084190    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084192    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084195    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084197    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084200    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084202    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084205    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084208    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084210    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084212    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084215    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084217    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084220    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084222    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084225    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084227    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084230    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.096209 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084232    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084236    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084238    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084241    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084244    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084246    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084249    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084251    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084254    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084256    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084259    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084261    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084265    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084268    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084271    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084274    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084276    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084279    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084281    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.096829 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084284    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084287    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084289    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084291    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084294    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084296    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084299    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084301    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084304    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084306    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084309    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084311    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084314    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084316    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084318    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084321    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084324    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084326    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084329    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084332    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.097270 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084334    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084337    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084339    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084342    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084344    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084347    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084349    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.084376    6738 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084528    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084533    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084536    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084539    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084541    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084545    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084547    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084551    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.097812 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084555    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084558    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084562    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084565    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084567    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084570    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084573    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084575    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084578    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084580    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084583    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084585    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084588    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084590    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084593    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084595    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084598    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084600    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084603    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084605    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.098201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084607    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084610    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084612    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084615    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084617    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084620    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084622    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084625    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084628    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084630    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084633    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084636    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084639    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084641    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084644    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084646    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084649    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084651    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084653    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084656    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.098724 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084658    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084661    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084663    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084666    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084669    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084671    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084674    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084676    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084678    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084681    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084683    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084686    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084688    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084690    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084693    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084695    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084698    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084700    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084703    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084706    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.099204 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084708    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084710    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084713    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084717    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084722    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084725    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084727    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084730    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084732    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084735    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084737    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084740    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084742    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084745    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084747    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084750    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084752    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084755    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.099950 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084757    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084760    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084762    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084765    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084767    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084769    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084772    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084774    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084777    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084779    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084782    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084784    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084787    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084789    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084791    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084794    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084797    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084799    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084802    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.100529 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084805    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.084828    6738 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084861    6738 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084865    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084868    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084871    6738 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084874    6738 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084876    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084879    6738 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084881    6738 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084883    6738 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084886    6738 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084893    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084896    6738 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084898    6738 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084901    6738 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 21:02:23.101064 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084903    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084906    6738 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084908    6738 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084911    6738 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084913    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084916    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084918    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084921    6738 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084924    6738 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084926    6738 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084929    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084931    6738 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084934    6738 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084936    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084939    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084941    6738 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084944    6738 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084947    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084950    6738 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 21:02:23.101629 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084955    6738 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084957    6738 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084960    6738 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084963    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084965    6738 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084968    6738 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084970    6738 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084972    6738 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084975    6738 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084979    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084981    6738 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084984    6738 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084987    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084989    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084992    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084994    6738 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084997    6738 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.084999    6738 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 21:02:23.102263 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085002    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085004    6738 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085007    6738 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085010    6738 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085012    6738 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085015    6738 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085017    6738 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085019    6738 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085022    6738 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085024    6738 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085027    6738 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085029    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085033    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085035    6738 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085039    6738 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085041    6738 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085044    6738 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085046    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085049    6738 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085051    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 21:02:23.102780 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085054    6738 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085056    6738 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085059    6738 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085063    6738 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085066    6738 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085069    6738 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085071    6738 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085074    6738 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085076    6738 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085079    6738 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085081    6738 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085083    6738 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085086    6738 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085088    6738 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085090    6738 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085094    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085096    6738 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085099    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085101    6738 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085104    6738 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 21:02:23.103253 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085106    6738 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085109    6738 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085111    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085113    6738 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085116    6738 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085119    6738 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085121    6738 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085124    6738 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085126    6738 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085129    6738 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085133    6738 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085135    6738 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085137    6738 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085140    6738 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:23.085142    6738 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.085199    6738 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086207    6738 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086265    6738 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086457    6738 server.go:1014] "Starting client certificate rotation"
Oct 30 21:02:23.103875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086618    6738 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086668    6738 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 20:56:13 +0000 UTC" deadline="2028-07-05 13:00:29.663806876 +0000 UTC"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086704    6738 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14727h58m6.577105817s"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.086994    6738 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.088093    6738 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.089473    6738 log.go:25] "Validated CRI v1 runtime API"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.091825    6738 log.go:25] "Validated CRI v1 image API"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.092567    6738 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 21:02:23.104403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.094862    6738 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ec605715-8ce4-4785-aaeb-765a8204f407:/dev/nvme0n1p3 fd7421f5-b59d-41d1-ac59-a2e8e87156df:/dev/nvme0n1p4]
Oct 30 21:02:23.104753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.094876    6738 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5/userdata/shm major:0 minor:364 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/239e3a9301d09602cfebfe17039b2020c433914d534a211ad149492a53933982/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/239e3a9301d09602cfebfe17039b2020c433914d534a211ad149492a53933982/userdata/shm major:0 minor:254 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0/userdata/shm major:0 minor:344 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36/userdata/shm major:0 minor:64 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6235887f50a87b6e308b9a5350e330860034780a87777c44b9143d80c4df6af8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6235887f50a87b6e308b9a5350e330860034780a87777c44b9143d80c4df6af8/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f/userdata/shm major:0 minor:426 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6f3503acdfc475b7d0f58d7adaaf0c8a3434b2b9a610d947231e8442c70f262e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6f3503acdfc475b7d0f58d7adaaf0c8a3434b2b9a610d947231e8442c70f262e/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743/userdata/shm major:0 minor:250 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2/userdata/shm major:0 minor:484 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb/userdata/shm major:0 minor:292 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0/userdata/shm major:0 minor:476 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955/userdata/shm major:0 minor:475 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6/userdata/shm major:0 minor:324 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~projected/kube-api-access-8w26g:{mountpoint:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~projected/kube-api-access-8w26g major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:244 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~projected/kube-api-access-99jdp:{mountpoint:/var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~projected/kube-api-access-99jdp major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~projected/kube-api-access-kn2qz:{mountpoint:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~projected/kube-api-access-kn2qz major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/config-reloader/2 major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/prometheus/15 major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~empty-dir/config-out major:0 minor:408 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/kube-api-access-9vgkk:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/kube-api-access-9vgkk major:0 minor:425 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/tls-assets major:0 minor:424 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/config major:0 minor:416 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:421 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:418 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:417 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:419 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:422 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:409 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:420 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/web-config major:0 minor:423 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d19a8f3-e037-4087-8d58-f1e05facef69/volumes/kubernetes.io~projected/kube-api-access-9jcnw:{mountpoint:/var/lib/kubelet/pods/4d19a8f3-e037-4087-8d58-f1e05facef69/volumes/kubernetes.io~projected/kube-api-access-9jcnw major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~projected/kube-api-access-w665x:{mountpoint:/var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~projected/kube-api-access-w665x major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/503db7ff-86df-4655-99e4-27d6db14c835/volumes/kubernetes.io~projected/kube-api-access-bpk5q:{mountpoint:/var/lib/kubelet/pods/503db7ff-86df-4655-99e4-27d6db14c835/volumes/kubernetes.io~projected/kube-api-access-bpk5q major:0 minor:479 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60535067-d426-468e-ab28-84675493491d/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/60535067-d426-468e-ab28-84675493491d/volumes/kubernetes.io~secret/agent-certs major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6f9a963b-963f-4195-9041-faa91154dc88/volumes/kubernetes.io~projected/kube-api-access-8stgg:{mountpoint:/var/lib/kubelet/pods/6f9a963b-963f-4195-9041-faa91154dc88/volumes/kubernetes.io~projected/kube-api-access-8stgg major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~projected/kube-api-access-z67ph:{mountpoint:/var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~projected/kube-api-access-z67ph major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~secret/metrics-tls major:0 minor:196 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~projected/kube-api-access-jwp8v:{mountpoint:/var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~projected/kube-api-access-jwp8v major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~secret/metrics-certs major:0 minor:474 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8e34a640-bd21-49d6-a1de-487db5d06085/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/8e34a640-bd21-49d6-a1de-487db5d06085/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:470 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~projected/kube-api-access-xxlxg:{mountpoint:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~projected/kube-api-access-xxlxg major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~projected/kube-api-access-58sqv:{mountpoint:/var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~projected/kube-api-access-58sqv major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:200 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~projected/kube-api-access-phdsl:{mountpoint:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~projected/kube-api-access-phdsl major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~projected/kube-api-access-n8fh2:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~projected/kube-api-access-n8fh2 major:0 minor:323 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:314 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:318 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c2020321-9a3d-4786-b6cb-f00ad50011be/volumes/kubernetes.io~projected/kube-api-access-x2zwb:{mountpoint:/var/lib/kubelet/pods/c2020321-9a3d-4786-b6cb-f00ad50011be/volumes/kubernetes.io~projected/kube-api-access-x2zwb major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~projected/kube-api-access-d5zfs:{mountpoint:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~projected/kube-api-access-d5zfs major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~projected/kube-api-access-42vnz:{mountpoint:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~projected/kube-api-access-42vnz major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:336 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:335 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7/volumes/kubernetes.io~projected/kube-api-access-m5h67:{mountpoint:/var/lib/kubelet/pods/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7/volumes/kubernetes.io~projected/kube-api-access-m5h67 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e86b21ac-fc11-4f42-a3de-db0bda5e47f6/volumes/kubernetes.io~projected/kube-api-access-47dpt:{mountpoint:/var/lib/kubelet/pods/e86b21ac-fc11-4f42-a3de-db0bda5e47f6/volumes/kubernetes.io~projected/kube-api-access-47dpt major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6/volumes/kubernetes.io~projected/kube-api-access-qhc74:{mountpoint:/var/lib/kubelet/pods/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6/volumes/kubernetes.io~projected/kube-api-access-qhc74 major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/alertmanager/9 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/config-reloader/6 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~empty-dir/config-out major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/kube-api-access-7rphv:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/kube-api-access-7rphv major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/tls-assets major:0 minor:287 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/config-volume major:0 minor:286 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/web-config major:0 minor:289 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/e7d3c9394ebe76d82367a75f03571cf5180dc6bc6a570e68914a912c2d3b2d37/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/4e047f59210a22d46a73f7cab89b5ca2646b4f58db6f9d107b5906d57a11eee0/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/1c7668af662763727e705d87ea1e2efa2dbc9b6902b482d6e367eb7c127076e9/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/f59afd355368d6ec26f355700f4c2b8a0a09d520c1ae831239484af327d85a7d/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/89c837559e0e39edc969caac1b52fa78a4f3d33611ffb5268f3484f726d9b8d1/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8b43873de75bf8be836acc4ae71f082ae606b1f56fc4eaf76d5a65d738cdd6a5/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/109d247404ad601b87b86d9206076bd8f67d645c8c34a8dd30e58239c1f95e6a/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/14d8f573e39d45885c133b4da0ba87f1d99c25fd3e95ed895a23e23b375f3da4/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/bca9779e9035ef50cf4e1bf260cfe20152249b84b67c5fcfcf845f5b55607fc2/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/05802cfae093d21c306b2ffb3703d1228f06b2d2ebfb0b7424168380146976f0/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/f8a1fb8b11dcc4f716cd5a7c16f85545b7d877309240cb3e2273f7223b1b1c39/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/055277dfead5a87cf54b485c9887a55c309172239e999da79f79ae7f5496b87d/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/2630d35bb458efe27338de6420e0244ba5b6d66989767a84feab13fc089690ca/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/91be230cdb09710e85a0e4adc841c4b77c036010dfff8b933d30a1d488d90117/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/a1ba720ab23ebd913760bd85e70f23110c138d67db1e77b32664215da37f5d48/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/5313cee2f3f10d98c3e7f16a98e9cc77905d67c28cb085634253efadc1ff22e2/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/b3e27889e408370a3c9358b19b9ff2f28869c7611aed24c23b2ea29579188e47/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/9143297060bcf3b040eb524e1a623934387a11fec2611df80af9c28f3ed44217/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/5f74dbab83c100946f8356d9bc5001e085028137264eb0e7f81907bf043fb741/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/2e00a53921f867db6d37bf16a8ae6d076e79a4d056291a93b40617d6e72da7e3/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/beec6e1eff86d2f4b2430d89f74302da131e92be5c22380c263d3858196510e2/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/1ba45bcc51d8b42d74462836d198233210fa38ce2004c1d9826d24c2bcf578f9/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/011c22494862c33822be3222854f2e7144782e3be6312af14def9e4a5ed7dff8/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/d9bca327dafcd72e3900e1831a2a9c8fd5e5f1c220544abe7e24d1bdaea4e8e9/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/cd47c46896c70559f9065fd28c7a48808d14f6febaf402c48bc621148dd68091/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/f2e0a286684a0080f2eff5dce7c2be422f06d64b4c27410db9ac1db632289b92/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/9526866c2969a779d74ebaf9364d0811115d8e413c31c0110264bfa37f1bd320/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/7683afd017cb402fab14d809c51d69bb3638c8dbab233a1c539347c3b1f21ff6/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/b78ba726e844fd175f85cf8b391e90dc90369bd8bf557d1e2c457992389e14f1/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/d3e360e26acebd53ee90cab9acf305721d33bc709fa38714d1fe4b6afb3285ac/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/6aa2cd521cb8736c439d24d13c2226a8cdbe1a85867e8dc16a2112e42a0491e8/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-261:{mountpoint:/var/lib/containers/storage/overlay/151550753d37c5e05e09d80d687e0d24cfde7c6a50ba6be46f451baa0ef5415b/merged major:0 minor:261 fsType:overlay blockSize:0} overlay_0-271:{mountpoint:/var/lib/containers/storage/overlay/b5dbd8819d79d6d7fdb0222ca186f3f54c1fedd7bc3b69c6bf3104109e936970/merged major:0 minor:271 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/101e9c2409299263fcf549936470578936a44cb1f9bf4ffd0163899e5fba3911/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-294:{mountpoint:/var/lib/containers/storage/overlay/88c924641f6dd697b7ad811c54010e0346d76e3cfe90c5d75e5aff4c0713ddc5/merged major:0 minor:294 fsType:overlay blockSize:0} overlay_0-296:{mountpoint:/var/lib/containers/storage/overlay/ed07bcfac9b6dd721592045958b637b24f691dea2274e7ff1d097541a99a65b6/merged major:0 minor:296 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/732c2f485b0e7fa06c70126582fea29ab0ad347389c20bf0d1058f67b23822a8/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/4475109feb0e27ab49ee2e4ec650716cdd2fbffdc3cd8ca68818af457d230b2f/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/a7db630413fe71f603c9618eb7c9b9d625bd1a2b10ab9831055e1f9cefa0eea7/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/2cf5a4add0d5ba2654de273cfb34a642b74d5a609c94e37cd2d77c8c80f09391/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/fc21d7a3754bcc9ec1121eb7e5c115929ed41d1977a480411e2fb60e831ab570/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/734ea950d80007287324d941612fe787208b1210c6b6cafee77b692dc5fdf2f8/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/7f865c79745738e66708e76435fe8c5d9eefa47d253b37d731ef3f07b0d3bd86/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-349:{mountpoint:/var/lib/containers/storage/overlay/c90166a6a52fb5ff4df8ca885ce096fd1a356d1c2bb5cafade5cd203ea7f5ca4/merged major:0 minor:349 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/a5e1e78d3042ea339f07aa04035149270fe05fa7b638f5abce806c129ca02b1c/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/354656081c6bcf0ba8c7d4f24640deb00ded721615b03dcac776c505d3401353/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/250d4f0076c9c3829371877c33962eaea9ccda7f4fa29728ee36beb9ff1df654/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/f253f4beae49d712e424cd92eea23896eb712fb8a057196485e81693388e5455/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/57ff055ad201dbb3416976360fc63ed5b0703b8f1f7666819dd721c238a27e47/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/d667b97adf8a936f8a2850a3fbe76da2e9efb7dd728b659e9337023a2f1147f2/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/c341c0f8aa543d1040ad558124190741d0d721d248e543f6ca9998d6201e14f3/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-386:{mountpoint:/var/lib/containers/storage/overlay/ce52f528414b491a0389a670b31a305efd913d62601f6cd21fcaaa64192dc2da/merged major:0 minor:386 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/721c066b32f558f1e54662f5050181fbcf53b0c4099ca3d32594d276c81b2339/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-396:{mountpoint:/var/lib/containers/storage/overlay/c37d468d9a726408b74f1bfe26b7ec61e5a63d797186ceffec8e60c5c1f1c1f8/merged major:0 minor:396 fsType:overlay blockSize:0} overlay_0-401:{mountpoint:/var/lib/containers/storage/overlay/3f5dcc500780facb5b733f5bb77035aa72e1376edb49c8a8a6b747f1a0c63126/merged major:0 minor:401 fsType:overlay blockSize:0} overlay_0-406:{mountpoint:/var/lib/containers/storage/overlay/79f1fee7d6adc88a4ba4b9456c8037a214b682b6d5acea58fbc8048dfede49c2/merged major:0 minor:406 fsType:overlay blockSize:0} overlay_0-428:{mountpoint:/var/lib/containers/storage/overlay/b45b3bd528f10cec76beaa6516cd4d20ab9b85fc0dd0d3da4cf3cd8b1bf1c65b/merged major:0 minor:428 fsType:overlay blockSize:0} overlay_0-430:{mountpoint:/var/lib/containers/storage/overlay/d7a2d1326230755c8f7c2a93188c9d4d9595d5544dc0315582bdbde5ea677039/merged major:0 minor:430 fsType:overlay blockSize:0} overlay_0-436:{mountpoint:/var/lib/containers/storage/overlay/fa958d3e32ab094f94924abfc0c92c53b9faf0818c61e65988c5c901a5fd1355/merged major:0 minor:436 fsType:overlay blockSize:0} overlay_0-438:{mountpoint:/var/lib/containers/storage/overlay/d0facac806b234e6085d8db23a3645482687ef660880e5f03da55e878b237687/merged major:0 minor:438 fsType:overlay blockSize:0} overlay_0-440:{mountpoint:/var/lib/containers/storage/overlay/8cc01226cfb42f5856f54deafb1b13522bef79b9f9eef4a5bca0e402e5ce3d57/merged major:0 minor:440 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/71d3bde04f62df794658460097cbf0e39d91fdb876628109fb6edda52a98b2de/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/be89261ee3d8971c6259b7aa94119da283fa0112fbad654c7b7c1b79a5fe1266/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-455:{mountpoint:/var/lib/containers/storage/overlay/36c54ffe33124aaf333d2d83bb2faed52089b82f263e41b0d9166d9a1934f09a/merged major:0 minor:455 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/ed8bd8ba03178fb198c25e83bbce7e86c5ae3719fc2189f9b06f010402473b7a/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-465:{mountpoint:/var/lib/containers/storage/overlay/4480925bb4c26b374287a5f3b7411494e3ee44afb6c9c406a16658682a2e59ab/merged major:0 minor:465 fsType:overlay blockSize:0} overlay_0-480:{mountpoint:/var/lib/containers/storage/overlay/934f87b4ec1ca20888e30b0ffbecab34f9a3c3ed50be89af226c6d3f10c93189/merged major:0 minor:480 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/aa6cdec89feedd76a8f93ea99ba005e55f2e4f912a8474afb11647fdb4e74304/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/0381fa8cf44e59d191fcde3dacc276d65a065ee4cd2aeb140402cfddf500acdc/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-488:{mountpoint:/var/lib/containers/storage/overlay/be516fc2dcbefd7f1c7e6608ee14b809a5c3df93e5f58fd20097f0af0e926e2d/merged major:0 minor:488 fsType:overlay blockSize:0} overlay_0-490:{mountpoint:/var/lib/containers/storage/overlay/8053f764fdb8d1af8c8da519e7ac27c6fb7042a373ae0808578838e7b3e2f0f2/merged major:0 minor:490 fsType:overlay blockSize:0} overlay_0-496:{mountpoint:/var/lib/containers/storage/overlay/365c6caa3b9d462c2b9665fb1d658b80bf02aee6d48f4661212056590b9e67b9/merged major:0 minor:496 fsType:overlay blockSize:0} overlay_0-501:{mountpoint:/var/lib/containers/storage/overlay/e9581df9a72b609be912753c0796a2f5f509a50cd65d22d22fd3fdbfd64c99aa/merged major:0 minor:501 fsType:overlay blockSize:0} overlay_0-62:{mountpoint:/var/lib/containers/storage/overlay/ffb1814cdb1b80c1e80298fb3f5be76b12c9419eeaa34647d13918b13b8ecdd3/merged major:0 minor:62 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/c99059e9e1a51bc22e6b597d7f9d86be06f8bb963d118c1641b058ee4742e0b6/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/2f80ab0dda342988f08a7cfaf719b2dc573f721b7fcb45a35f103f538da46847/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/c6c279270dc9d918e28d3f8f5002def9bb0c8c08f835517dcace0e0127fa48e4/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/84bd5e0173702a317658bfc50c27222837d653d674da7226f9f3c7cb9d61c05f/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/2bed27356eca12da67d888fc53aca5bf0c880cd14dfac0695421e6cbf11242ba/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/e611733cd05382d0b7d5428363744b21243c181b6e500f11a3acc62536dbd20e/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/6cbbb711fd65f699a99968ad7a602dec4208724eaa5fcd5435cfe445db23fe1a/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/52d939b6ff82b4b33e80d318ceb106883ffb48634b08ce68ad2ef165a50f0472/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/86a7d57f3b7b157a6eb363476ff992e995aef145c76676e4373e84fb88734f90/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.103595    6738 manager.go:217] Machine: {Timestamp:2026-10-30 21:02:23.103120472 +0000 UTC m=+0.058807697 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3133707 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ae5838e695c1881ab41fb4c9d9ca2 SystemUUID:ec2ae583-8e69-5c18-81ab-41fb4c9d9ca2 BootID:3528c481-b330-4707-bcd4-c54c62f1d00d Filesystems:[{Device:overlay_0-386 DeviceMajor:0 DeviceMinor:386 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-406 DeviceMajor:0 DeviceMinor:406 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:416 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-480 DeviceMajor:0 DeviceMinor:480 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~projected/kube-api-access-8w26g DeviceMajor:0 DeviceMinor:239 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-455 DeviceMajor:0 DeviceMinor:455 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/681e40923a4fb83f334c5d46d2f7d6f954863db9fbe89419371c74747d8407c6/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-430 DeviceMajor:0 DeviceMinor:430 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/e86b21ac-fc11-4f42-a3de-db0bda5e47f6/volumes/kubernetes.io~projected/kube-api-access-47dpt DeviceMajor:0 DeviceMinor:54 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:423 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-501 DeviceMajor:0 DeviceMinor:501 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3fcb77f29b089a145e35801e4e75a51ec564b3af346726b6a8a8d87c2d2be237/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~projected/kube-api-access-58sqv DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6f3503acdfc475b7d0f58d7adaaf0c8a3434b2b9a610d947231e8442c70f262e/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ae7d66155969ab2c8b621710692a67e51e2f85286f5e040cc719c9c4a8103dfd/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:286 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~projected/kube-api-access-w665x DeviceMajor:0 DeviceMinor:363 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:420 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8e34a640-bd21-49d6-a1de-487db5d06085/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:470 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-62 DeviceMajor:0 DeviceMinor:62 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36/userdata/shm DeviceMajor:0 DeviceMinor:64 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:243 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-261 DeviceMajor:0 DeviceMinor:261 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:408 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-271 DeviceMajor:0 DeviceMinor:271 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:290 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb/userdata/shm DeviceMajor:0 DeviceMinor:292 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:409 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:421 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/60535067-d426-468e-ab28-84675493491d/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:48 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/09d9a38cb535018c8b2e038ca770238815b1ef60e8e531dfc3a64913aeca66c0/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-296 DeviceMajor:0 DeviceMinor:296 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f/userdata/shm DeviceMajor:0 DeviceMinor:426 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1699b579-56d9-40a2-a5c5-8253c97fd0d8/volumes/kubernetes.io~projected/kube-api-access-99jdp DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:318 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:419 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:424 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-436 DeviceMajor:0 DeviceMinor:436 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/503db7ff-86df-4655-99e4-27d6db14c835/volumes/kubernetes.io~projected/kube-api-access-bpk5q DeviceMajor:0 DeviceMinor:479 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:337 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~projected/kube-api-access-jwp8v DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~projected/kube-api-access-kn2qz DeviceMajor:0 DeviceMinor:242 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-349 DeviceMajor:0 DeviceMinor:349 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7d5e057f14e170ac3267f6686db36d1892f101e7585092f146eb47e93c079970/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:244 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cc0de8eceb0f16d01d108625827510cfab2506bfd277fa1ca4dc8ac4be6cbdfc/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-490 DeviceMajor:0 DeviceMinor:490 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~projected/kube-api-access-phdsl DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:200 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f167d0cf10c079216c0c71ad00e4dbada181e15791161c6b3c332c1c5a991fa2/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6f9a963b-963f-4195-9041-faa91154dc88/volumes/kubernetes.io~projected/kube-api-access-8stgg DeviceMajor:0 DeviceMinor:53 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b5167fe841047293ed7632371b5e5ed2a1d524cf63eef55bcd40c844af0e3955/userdata/shm DeviceMajor:0 DeviceMinor:475 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a209dc6c-34cd-4fd3-90f2-2591f28988a8/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:50 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~projected/kube-api-access-z67ph DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6235887f50a87b6e308b9a5350e330860034780a87777c44b9143d80c4df6af8/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:278 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:236 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:335 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~projected/kube-api-access-42vnz DeviceMajor:0 DeviceMinor:338 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/02d623231353eee74e63645704348bb1ee2ce7df6407f316710ee9e124c314b5/userdata/shm DeviceMajor:0 DeviceMinor:364 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-438 DeviceMajor:0 DeviceMinor:438 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d19a8f3-e037-4087-8d58-f1e05facef69/volumes/kubernetes.io~projected/kube-api-access-9jcnw DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8efe43a54f171990c69d4a4b32e9bfbe00d03801ae8dab78956805f834a85743/userdata/shm DeviceMajor:0 DeviceMinor:250 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d3abab98-e4d8-4ebd-b4a3-81d91079ab72/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:336 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:418 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-428 DeviceMajor:0 DeviceMinor:428 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/068a733e-f3f0-4aec-9db1-a78b9e81c332/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:238 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:314 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-294 DeviceMajor:0 DeviceMinor:294 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:422 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/24149cb7fa545928b8502738beebec300e7bfb94f05a004dc9bd0ff2d512d3b0/userdata/shm DeviceMajor:0 DeviceMinor:344 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:423 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/acfcff9511679eb3c97b08a390796a25e32dc678e52576e4809e2b5989c4bfa0/userdata/shm DeviceMajor:0 DeviceMinor:476 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7/volumes/kubernetes.io~projected/kube-api-access-m5h67 DeviceMajor:0 DeviceMinor:51 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-396 DeviceMajor:0 DeviceMinor:396 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-488 DeviceMajor:0 DeviceMinor:488 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:288 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c2020321-9a3d-4786-b6cb-f00ad50011be/volumes/kubernetes.io~projected/kube-api-access-x2zwb DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:241 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:423 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc/volumes/kubernetes.io~projected/kube-api-access-d5zfs DeviceMajor:0 DeviceMinor:52 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7c206b38791ea9ecf497fcfb4a7bceb5e46ba1e08e90a80212f09553678dff72/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:285 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:287 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:320 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~projected/kube-api-access-n8fh2 DeviceMajor:0 DeviceMinor:323 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/152f1e403b8cd14673f07e6f764973ce984b93529f31a57cd9d62dab187c4c62/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:284 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:321 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-401 DeviceMajor:0 DeviceMinor:401 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81a6d809-1b40-4272-9366-466dec6031cf/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:474 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1455da1e3e2df5b762c325f3acad752147285e3fda68c1fd0d0ed90ed38a0b2/userdata/shm DeviceMajor:0 DeviceMinor:484 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6/volumes/kubernetes.io~projected/kube-api-access-qhc74 DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:237 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/239e3a9301d09602cfebfe17039b2020c433914d534a211ad149492a53933982/userdata/shm DeviceMajor:0 DeviceMinor:254 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:283 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:319 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b78874e1-9201-4cfa-871f-1af75327111b/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:322 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ef196e92a6a99192b9eb6ab63b29d7524295c096447be96936125ab1144214b6/userdata/shm DeviceMajor:0 DeviceMinor:324 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4fa94833-b283-4190-b623-f36b7c9edf2f/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:362 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:417 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes/kubernetes.io~projected/kube-api-access-9vgkk DeviceMajor:0 DeviceMinor:425 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:284 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:423 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-465 DeviceMajor:0 DeviceMinor:465 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ad08418-9a79-4912-831e-d49f5802f695/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:196 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/volumes/kubernetes.io~projected/kube-api-access-xxlxg DeviceMajor:0 DeviceMinor:240 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-440 DeviceMajor:0 DeviceMinor:440 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-496 DeviceMajor:0 DeviceMinor:496 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e816af491fb2b604200975b4c16a2d2e1e83c786391f741c17eb201686e84ee2/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes/kubernetes.io~projected/kube-api-access-7rphv DeviceMajor:0 DeviceMinor:291 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:02d623231353eee MacAddress:42:8f:8d:17:81:72 Speed:10000 Mtu:8901} {Name:239e3a9301d0960 MacAddress:ee:78:36:b2:fd:f0 Speed:10000 Mtu:8901} {Name:24149cb7fa54592 MacAddress:da:68:43:ff:20:e2 Speed:10000 Mtu:8901} {Name:6235887f50a87b6 MacAddress:a2:05:e0:90:1b:3c Speed:10000 Mtu:8901} {Name:640438d0e28af12 MacAddress:0a:c4:9f:c1:ef:cb Speed:10000 Mtu:8901} {Name:6f3503acdfc475b MacAddress:16:72:8a:35:fb:ea Speed:10000 Mtu:8901} {Name:a1455da1e3e2df5 MacAddress:52:ec:5b:e7:39:af Speed:10000 Mtu:8901} {Name:a7d93b2a43c77ff MacAddress:6e:fb:24:1c:64:76 Speed:10000 Mtu:8901} {Name:acfcff9511679eb MacAddress:7e:7d:1e:27:78:73 Speed:10000 Mtu:8901} {Name:b5167fe84104729 MacAddress:6e:2f:d6:1d:65:b6 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:ee:71:9f:8d Speed:0 Mtu:9001} {Name:br-int MacAddress:12:72:8f:41:55:fb Speed:0 Mtu:8901} {Name:cc0de8eceb0f16d MacAddress:da:b4:0b:9b:b5:92 Speed:10000 Mtu:8901} {Name:ef196e92a6a9919 MacAddress:ea:6b:4b:62:5d:26 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:ee:71:9f:8d Speed:0 Mtu:9001} {Name:f167d0cf10c0792 MacAddress:2e:c2:1c:3a:6b:82 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:46:68:16:c7:31:99 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:c6:b5:03:be:e5:17 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104044    6738 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104088    6738 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104340    6738 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104366    6738 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-2.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104502    6738 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104510    6738 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104520    6738 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104538    6738 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104616    6738 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104765    6738 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104868    6738 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104901    6738 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104912    6738 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104922    6738 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104931    6738 kubelet.go:405] "Adding apiserver pod source"
Oct 30 21:02:23.105077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.104938    6738 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 21:02:23.105987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.105970    6738 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:02:23.106023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.105990    6738 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:02:23.106497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.106478    6738 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 21:02:23.109015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.108997    6738 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 21:02:23.109015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109013    6738 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 21:02:23.109183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109120    6738 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 21:02:23.109350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109335    6738 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 21:02:23.109411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109375    6738 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 21:02:23.109519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109507    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109527    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109536    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109545    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109554    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109564    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 21:02:23.109579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109574    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109589    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109601    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109610    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109623    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109639    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109673    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 21:02:23.109946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.109681    6738 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 21:02:23.113640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.113621    6738 server.go:1263] "Started kubelet"
Oct 30 21:02:23.113833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.113758    6738 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 21:02:23.113903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.113869    6738 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 21:02:23.114250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.114232    6738 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 21:02:23.114389 ip-10-0-139-2 systemd[1]: Started Kubernetes Kubelet.
Oct 30 21:02:23.114566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.113673    6738 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 21:02:23.116424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.116409    6738 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 21:02:23.120092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120075    6738 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 21:02:23.120228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120091    6738 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:02:23.120306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120160    6738 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 21:02:23.120306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120270    6738 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 21:02:23.120306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120284    6738 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 21:02:23.120482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120292    6738 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 20:56:14 +0000 UTC" deadline="2028-06-13 11:04:57.657474437 +0000 UTC"
Oct 30 21:02:23.120482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120326    6738 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14198h2m34.537151776s"
Oct 30 21:02:23.120482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.120391    6738 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 21:02:23.121352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.121332    6738 factory.go:55] Registering systemd factory
Oct 30 21:02:23.121352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.121353    6738 factory.go:223] Registration of the systemd container factory successfully
Oct 30 21:02:23.122288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122265    6738 factory.go:153] Registering CRI-O factory
Oct 30 21:02:23.122365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122291    6738 factory.go:223] Registration of the crio container factory successfully
Oct 30 21:02:23.122365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122363    6738 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 21:02:23.122491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122383    6738 factory.go:103] Registering Raw factory
Oct 30 21:02:23.122491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122399    6738 manager.go:1199] Started watching for new ooms in manager
Oct 30 21:02:23.122775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122755    6738 manager.go:319] Starting recovery of all containers
Oct 30 21:02:23.122858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.122816    6738 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:02:23.123958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.123933    6738 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:02:23.124958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124284    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124957    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124973    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124982    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124991    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fa94833-b283-4190-b623-f36b7c9edf2f" volumeName="kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.124999    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ad08418-9a79-4912-831e-d49f5802f695" volumeName="kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125007    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ad08418-9a79-4912-831e-d49f5802f695" volumeName="kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125016    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125026    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125033    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125041    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc" volumeName="kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125049    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125056    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e86b21ac-fc11-4f42-a3de-db0bda5e47f6" volumeName="kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125066    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068a733e-f3f0-4aec-9db1-a78b9e81c332" volumeName="kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:02:23.125070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125078    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125086    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125094    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125103    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81a6d809-1b40-4272-9366-466dec6031cf" volumeName="kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125110    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc" volumeName="kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125119    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125126    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125147    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125155    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f9a963b-963f-4195-9041-faa91154dc88" volumeName="kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125164    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125172    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125181    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e86b21ac-fc11-4f42-a3de-db0bda5e47f6" volumeName="kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125188    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125205    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125217    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125225    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125232    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f9a963b-963f-4195-9041-faa91154dc88" volumeName="kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125242    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ad08418-9a79-4912-831e-d49f5802f695" volumeName="kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125250    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125257    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125265    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1699b579-56d9-40a2-a5c5-8253c97fd0d8" volumeName="kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125272    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125280    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125288    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125296    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125305    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="503db7ff-86df-4655-99e4-27d6db14c835" volumeName="kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125313    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f9a963b-963f-4195-9041-faa91154dc88" volumeName="kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125321    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c" volumeName="kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125328    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125337    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125345    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125353    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125360    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125367    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125375    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ad08418-9a79-4912-831e-d49f5802f695" volumeName="kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125383    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125391    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e86b21ac-fc11-4f42-a3de-db0bda5e47f6" volumeName="kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125398    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d19a8f3-e037-4087-8d58-f1e05facef69" volumeName="kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125407    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6f9a963b-963f-4195-9041-faa91154dc88" volumeName="kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125416    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125521    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c" volumeName="kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125538    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a209dc6c-34cd-4fd3-90f2-2591f28988a8" volumeName="kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125557    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a209dc6c-34cd-4fd3-90f2-2591f28988a8" volumeName="kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125571    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a209dc6c-34cd-4fd3-90f2-2591f28988a8" volumeName="kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125580    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125588    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125596    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125604    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125612    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125619    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6" volumeName="kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125628    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125638    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125645    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125653    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125661    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c2020321-9a3d-4786-b6cb-f00ad50011be" volumeName="kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125669    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c2020321-9a3d-4786-b6cb-f00ad50011be" volumeName="kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125676    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc" volumeName="kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125683    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 21:02:23.125754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.125693    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068a733e-f3f0-4aec-9db1-a78b9e81c332" volumeName="kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.126649    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127874    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60535067-d426-468e-ab28-84675493491d" volumeName="kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127906    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8e34a640-bd21-49d6-a1de-487db5d06085" volumeName="kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127926    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127948    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127961    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.127981    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068a733e-f3f0-4aec-9db1-a78b9e81c332" volumeName="kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128002    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4fa94833-b283-4190-b623-f36b7c9edf2f" volumeName="kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128017    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c" volumeName="kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128038    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128052    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128069    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60535067-d426-468e-ab28-84675493491d" volumeName="kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128082    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a209dc6c-34cd-4fd3-90f2-2591f28988a8" volumeName="kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128098    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b78874e1-9201-4cfa-871f-1af75327111b" volumeName="kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128115    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e6a6b03c-0628-4bea-bbaa-66f0c0108dd7" volumeName="kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128128    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128146    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128159    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d19a8f3-e037-4087-8d58-f1e05facef69" volumeName="kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128175    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81a6d809-1b40-4272-9366-466dec6031cf" volumeName="kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128198    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6" volumeName="kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128211    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f666bd4a-175a-42dd-adb8-11aaf725c174" volumeName="kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128230    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1699b579-56d9-40a2-a5c5-8253c97fd0d8" volumeName="kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128243    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128261    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128274    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7" volumeName="kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128289    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c" volumeName="kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 21:02:23.128294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128308    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a209dc6c-34cd-4fd3-90f2-2591f28988a8" volumeName="kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128321    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128338    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d3abab98-e4d8-4ebd-b4a3-81d91079ab72" volumeName="kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128351    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="068a733e-f3f0-4aec-9db1-a78b9e81c332" volumeName="kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128364    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" volumeName="kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128384    6738 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4acaeddc-b0dd-451a-8cbe-353b0a0823af" volumeName="kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128396    6738 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 21:02:23.129618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.128405    6738 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 21:02:23.132687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.132669    6738 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 21:02:23.154245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.154212    6738 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 21:02:23.155921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.155898    6738 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 21:02:23.155921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.155922    6738 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 21:02:23.156257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.155944    6738 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 21:02:23.156257 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.155981    6738 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 21:02:23.159862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.159832    6738 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 21:02:23.180825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.180792    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e" exitCode=0
Oct 30 21:02:23.185938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.185908    6738 generic.go:356] "Generic (PLEG): container finished" podID="e8b94293ba553932730b321b7e106581" containerID="bd734250ab4eeb48da9160bb9058e91ee69e8aca840c798f3410404e0edbe789" exitCode=0
Oct 30 21:02:23.200265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200233    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="1b56ce1bc45d7d265c2e4c3aae89572a176db167430818019dcdb1f6810352b8" exitCode=0
Oct 30 21:02:23.200265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200263    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="8e99b664fbc430536985dfe6604b43f16fb86e7e566302afbdffc998a5b43f5c" exitCode=0
Oct 30 21:02:23.200429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200276    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="b29687b67f36e99cf1776d39e87d382e979f957918b6dabb78d9fa77a9c4edd0" exitCode=0
Oct 30 21:02:23.200429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200289    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="c5d1546496764a438bef9b01c74f6a9949b2fa55e86f94de678a06afb7aa753b" exitCode=0
Oct 30 21:02:23.200429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200301    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="04a14c1949e883f3760fe09aebb3eaf8ead24b837c50bc1d9dfe2b9e030297b6" exitCode=0
Oct 30 21:02:23.200429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.200314    6738 generic.go:356] "Generic (PLEG): container finished" podID="6f9a963b-963f-4195-9041-faa91154dc88" containerID="6e7570be09a79e284f54226ee2763a17c34034798b1da1c8b8917607b231080a" exitCode=0
Oct 30 21:02:23.213651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.213615    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a" exitCode=0
Oct 30 21:02:23.216111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.216086    6738 generic.go:356] "Generic (PLEG): container finished" podID="2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f" containerID="f8728e7d3cc8210218f34a4628d7bfa5b0a213426c9c0666de5f506c9dce6515" exitCode=0
Oct 30 21:02:23.241159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.241099    6738 manager.go:324] Recovery completed
Oct 30 21:02:23.256692 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.256660    6738 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 21:02:23.264105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264082    6738 cpu_manager.go:226] "Starting" policy="none"
Oct 30 21:02:23.264105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264097    6738 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 21:02:23.264237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264114    6738 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 21:02:23.264269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264241    6738 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 21:02:23.264269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264252    6738 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 21:02:23.264269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264268    6738 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 21:02:23.264356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264273    6738 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 21:02:23.264356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264279    6738 policy_none.go:50] "Start"
Oct 30 21:02:23.264356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264287    6738 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 21:02:23.264356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264296    6738 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 21:02:23.264486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264451    6738 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 21:02:23.264486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264460    6738 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 21:02:23.264486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.264466    6738 policy_none.go:44] "Start"
Oct 30 21:02:23.269261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.269235    6738 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 21:02:23.271992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.271977    6738 manager.go:341] "Starting Device Plugin manager"
Oct 30 21:02:23.272083 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.272001    6738 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 21:02:23.272083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.272011    6738 server.go:87] "Starting device plugin registration server"
Oct 30 21:02:23.272828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.272815    6738 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 21:02:23.272897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.272829    6738 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 21:02:23.272947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.272930    6738 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 21:02:23.273068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.273032    6738 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 21:02:23.273486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.273462    6738 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 21:02:23.273486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.273483    6738 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 21:02:23.273727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.273614    6738 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T21:02:23.27306657Z","UUID":"db4c22ca-ad9f-4ad7-b0dc-91c996c9e1a5","Handler":null,"Name":"","Endpoint":""}
Oct 30 21:02:23.275373 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.275349    6738 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 21:02:23.278496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.278420    6738 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 21:02:23.278602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.278499    6738 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 21:02:23.373788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.373737    6738 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 21:02:23.377348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.377328    6738 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 21:02:23.377416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.377365    6738 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 21:02:23.377416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.377381    6738 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-2.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 21:02:23.377508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.377465    6738 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.388882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.388865    6738 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.388935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.388928    6738 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.457264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.457187    6738 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"]
Oct 30 21:02:23.457966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.457952    6738 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.458028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.457956    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerStarted","Data":"0cca92fffc3c5e3c81db6614fddc99479c8eca8bcc7d0bc580cd40f5c72554eb"}
Oct 30 21:02:23.458028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.458000    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerDied","Data":"bd734250ab4eeb48da9160bb9058e91ee69e8aca840c798f3410404e0edbe789"}
Oct 30 21:02:23.458028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.458015    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"e8b94293ba553932730b321b7e106581","Type":"ContainerStarted","Data":"49d7ace36a6b1bf567868097c83bf7278636def38c3932beecf556d1fdf72f36"}
Oct 30 21:02:23.458120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.458064    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"f96f6cddab271c4d36b89844c0672b86","Type":"ContainerStarted","Data":"e01094799f58b9357f97d53c21880a290a7e556b3c2ee3edc317e19d939ffb11"}
Oct 30 21:02:23.458120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.458076    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal" event={"ID":"f96f6cddab271c4d36b89844c0672b86","Type":"ContainerStarted","Data":"2610dfb8313d1e17c138c81be019ae054eeea5166c78a399ab9715a7438f903e"}
Oct 30 21:02:23.458187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.458159    6738 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.470770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.470745    6738 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:02:23.470820 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.470793    6738 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.471289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.471273    6738 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:02:23.471339 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:23.471326    6738 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.534637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.534557    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.534637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.534594    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.534637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.534612    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635140    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635190    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635222    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635252    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635267    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f96f6cddab271c4d36b89844c0672b86-config\") pod \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"f96f6cddab271c4d36b89844c0672b86\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:23.635323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:23.635310    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8b94293ba553932730b321b7e106581-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" (UID: \"e8b94293ba553932730b321b7e106581\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:24.106049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.106014    6738 apiserver.go:52] "Watching apiserver"
Oct 30 21:02:24.117011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.116981    6738 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 21:02:24.117630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.117608    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-vzxd4","kube-system/global-pull-secret-syncer-26bcq","openshift-monitoring/monitoring-plugin-77f4f76594-72dpx","openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s","openshift-network-diagnostics/network-check-target-nkrzz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn","openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n","openshift-monitoring/prometheus-k8s-0","openshift-network-operator/iptables-alerter-f9fz4","kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-ctlvw","openshift-multus/multus-2zxxc","openshift-insights/insights-runtime-extractor-v6xnw","openshift-monitoring/metrics-server-8b68d8d9-5tfbm","kube-system/konnectivity-agent-m474j","openshift-dns/dns-default-jmdvh","openshift-monitoring/alertmanager-main-0","openshift-monitoring/node-exporter-srnnf","openshift-cluster-node-tuning-operator/tuned-pkswx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal","openshift-monitoring/thanos-querier-756469fbcb-g9zw7","openshift-ovn-kubernetes/ovnkube-node-c45gr","openshift-dns/node-resolver-6gb6m","openshift-ingress-canary/ingress-canary-hlskz","openshift-image-registry/node-ca-qkbt2"]
Oct 30 21:02:24.124063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124033    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-qd7pl\""
Oct 30 21:02:24.124190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124033    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x8hl8\""
Oct 30 21:02:24.124190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124085    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qfb4q\""
Oct 30 21:02:24.124190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124067    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 21:02:24.124190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124057    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-v7745\""
Oct 30 21:02:24.124190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124105    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mwc48\""
Oct 30 21:02:24.124422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124197    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 21:02:24.124422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124407    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-99mk5\""
Oct 30 21:02:24.124569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124049    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hspc2\""
Oct 30 21:02:24.124569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124554    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 21:02:24.124723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124598    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.124723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.124636    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 21:02:24.125394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125369    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qbtrh\""
Oct 30 21:02:24.125495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125385    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.125495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125407    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.125616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125547    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.125616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125589    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.125711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125626    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 21:02:24.125711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125636    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 21:02:24.125711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125642    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.125711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125684    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 21:02:24.125892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125718    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.125892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125845    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.125892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125874    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.126029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125915    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.126029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125925    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 21:02:24.126029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125951    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-6bj8t\""
Oct 30 21:02:24.126029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125959    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 21:02:24.126029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.125982    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 21:02:24.126241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.126212    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.126676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.126653    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 21:02:24.126676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.126683    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.126844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.126828    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 21:02:24.127018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.126993    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.127018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.127010    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6m5dr\""
Oct 30 21:02:24.127216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.127200    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.127819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.127803    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 21:02:24.127972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.127955    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.128293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128260    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.128374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128348    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 21:02:24.128430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128421    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-g68zh\""
Oct 30 21:02:24.128539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.127823    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 21:02:24.128711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128694    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.128774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128763    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 21:02:24.128998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.128980    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 21:02:24.129062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129048    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cp8br\""
Oct 30 21:02:24.129203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129180    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-s7wtw\""
Oct 30 21:02:24.129378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129361    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 21:02:24.129479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129463    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 21:02:24.129546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129532    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-gxjw5\""
Oct 30 21:02:24.129604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129577    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 21:02:24.129655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129622    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.129737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129724    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 21:02:24.129869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129728    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.129926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129865    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4f5fg\""
Oct 30 21:02:24.129971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129919    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-wx578\""
Oct 30 21:02:24.129971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.129958    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.130073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130060    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 21:02:24.130168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130153    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 21:02:24.130227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130165    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:02:24.130227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130202    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 21:02:24.130227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130154    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xpqzq\""
Oct 30 21:02:24.130485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130470    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 21:02:24.130559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130542    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 21:02:24.130683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130665    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 21:02:24.130742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.130728    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 21:02:24.133638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.133609    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 21:02:24.133638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.133622    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9b2tx\""
Oct 30 21:02:24.133778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.133658    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 21:02:24.134284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.134267    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:02:24.134612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.134598    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:02:24.137144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137126    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137149    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137169    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137184    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.137223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137202    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.137389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137238    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137353    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8stgg\" (UniqueName: \"kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.137389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137374    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2020321-9a3d-4786-b6cb-f00ad50011be-tmp-dir\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.137562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137384    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.137562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137389    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-config\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137421    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137505    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137528    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137614    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137631    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137646    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137663    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137709    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137671    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:02:24.137766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137734    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137767    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137780    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137766    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137830    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137831    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137891    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lgbjz\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137907    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137923    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137985    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.137998    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138004    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138016    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138041    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138055    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 21:02:24.138085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138003    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4d19a8f3-e037-4087-8d58-f1e05facef69-serviceca\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138140    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138183    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138221    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jcnw\" (UniqueName: \"kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138249    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138275    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovnkube-script-lib\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138273    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138320    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a209dc6c-34cd-4fd3-90f2-2591f28988a8-ovn-node-metrics-cert\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138407    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138479    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138507    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.138624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138621    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138650    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138682    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-iptables-alerter-script\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138713    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138718    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138759    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138807    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:02:24.138913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138872    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.138939    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139015    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 21:02:24.139140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139044    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-zxwnt\""
Oct 30 21:02:24.139216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139140    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/60535067-d426-468e-ab28-84675493491d-konnectivity-ca\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:02:24.139216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139181    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139233    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.139272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139243    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7dnlj6d6sts0u\""
Oct 30 21:02:24.139272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139266    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhc74\" (UniqueName: \"kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.139348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139284    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 21:02:24.139348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139308    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139322    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1t1guua5phs70\""
Oct 30 21:02:24.139348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139341    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:02:24.139477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139358    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2zwb\" (UniqueName: \"kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.139477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139392    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-2j8dz\""
Oct 30 21:02:24.139477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139367    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 21:02:24.139477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139412    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3ji7goluo3bfd\""
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139491    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139514    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dsnfd\""
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139514    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139391    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139574    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139604    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139609    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139621    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139623    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139655    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phdsl\" (UniqueName: \"kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139703    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139718    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139758    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139782    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6f9a963b-963f-4195-9041-faa91154dc88-cni-binary-copy\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139808    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5h67\" (UniqueName: \"kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139844    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a209dc6c-34cd-4fd3-90f2-2591f28988a8-env-overrides\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.139905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.139870    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140095    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/60535067-d426-468e-ab28-84675493491d-agent-certs\") pod \"konnectivity-agent-m474j\" (UID: \"60535067-d426-468e-ab28-84675493491d\") " pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140132    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140167    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140223    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140240    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:02:24.140265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140228    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:02:24.140499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140265    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:02:24.140499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140271    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:02:24.140499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.140371    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9ljfc\""
Oct 30 21:02:24.144860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.144835    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:02:24.146327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146266    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-phdsl\" (UniqueName: \"kubernetes.io/projected/a209dc6c-34cd-4fd3-90f2-2591f28988a8-kube-api-access-phdsl\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.146327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146284    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8stgg\" (UniqueName: \"kubernetes.io/projected/6f9a963b-963f-4195-9041-faa91154dc88-kube-api-access-8stgg\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.146327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146293    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5h67\" (UniqueName: \"kubernetes.io/projected/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kube-api-access-m5h67\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.146327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146321    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2zwb\" (UniqueName: \"kubernetes.io/projected/c2020321-9a3d-4786-b6cb-f00ad50011be-kube-api-access-x2zwb\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.146663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146644    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpk5q\" (UniqueName: \"kubernetes.io/projected/503db7ff-86df-4655-99e4-27d6db14c835-kube-api-access-bpk5q\") pod \"network-check-target-nkrzz\" (UID: \"503db7ff-86df-4655-99e4-27d6db14c835\") " pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:24.146729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146667    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jcnw\" (UniqueName: \"kubernetes.io/projected/4d19a8f3-e037-4087-8d58-f1e05facef69-kube-api-access-9jcnw\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.146729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146651    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:02:24.146729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.146710    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhc74\" (UniqueName: \"kubernetes.io/projected/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-kube-api-access-qhc74\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.222478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.222450    6738 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 21:02:24.223457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.223413    6738 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:24.224042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.223846    6738 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:24.234629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.234613    6738 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:02:24.234726 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:24.234657    6738 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:24.237580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.237556    6738 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 21:02:24.238272 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:24.237591    6738 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-2.us-east-2.compute.internal"
Oct 30 21:02:24.240112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240091    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99jdp\" (UniqueName: \"kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:02:24.240210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240120    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.240210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240140    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.240210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240155    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47dpt\" (UniqueName: \"kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.240210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240176    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.240210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240201    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxlxg\" (UniqueName: \"kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240253    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240277    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240277    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-etc-selinux\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240300    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240326    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240351    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w26g\" (UniqueName: \"kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240378    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-data-volume\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240381    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240409    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.240467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240450    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240479    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240505    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240532    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240534    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240558    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240588    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240591    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-tls\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240612    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240616    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240644    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240668    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240670    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-var-lib-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240696    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240704    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240616    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-ovn\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240771    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240784    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240843    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-client-certs\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240840    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240870    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-log-socket\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240927    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240952    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-client-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240957    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240952    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.240999    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241050    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.241103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241105    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241138    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241190    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241215    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241222    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241243    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241258    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-device-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241268    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241282    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-bin\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241290    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241297    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241325    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241349    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241374    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241397    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241404    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-tmp\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241422    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241469    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-kubelet\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241471    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241425    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b78874e1-9201-4cfa-871f-1af75327111b-metrics-client-ca\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241507    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241500    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241552    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241575    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ad08418-9a79-4912-831e-d49f5802f695-tmp-dir\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241576    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241578    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241616    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241632    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241644    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241646    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c2020321-9a3d-4786-b6cb-f00ad50011be-hosts-file\") pod \"node-resolver-6gb6m\" (UID: \"c2020321-9a3d-4786-b6cb-f00ad50011be\") " pod="openshift-dns/node-resolver-6gb6m"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241673    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-node-log\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241683    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z67ph\" (UniqueName: \"kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241709    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241733    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241906    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.241912    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242000    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242056    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242077    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242095    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242126    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4d19a8f3-e037-4087-8d58-f1e05facef69-host\") pod \"node-ca-qkbt2\" (UID: \"4d19a8f3-e037-4087-8d58-f1e05facef69\") " pod="openshift-image-registry/node-ca-qkbt2"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242133    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242180    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242207    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242234    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242261    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242285    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242311    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242330    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242345    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.242388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242403    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242459    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwp8v\" (UniqueName: \"kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242486    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242512    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242536    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242558    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242557    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242573    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242607    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242639    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242658    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-tuned\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242666    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242670    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242706    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242730    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242760    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242774    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242791    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242818    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242857    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242886    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1699b579-56d9-40a2-a5c5-8253c97fd0d8-cert\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242913    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-socket-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242950    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242978    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.242984    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-textfile\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243080    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-grpc-tls\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243180    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243191    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243212    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243238    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243251    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243257    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243262    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42vnz\" (UniqueName: \"kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243309    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243334    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-audit-log\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243307    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243335    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243399    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243468    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243483    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-daemon-config\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243496    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243523    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243555    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/81a6d809-1b40-4272-9366-466dec6031cf-metrics-certs\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243557    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-cni-netd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243560    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243594    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243607    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-registration-dir\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243762    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/068a733e-f3f0-4aec-9db1-a78b9e81c332-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243772    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243780    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243803    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-etc-openvswitch\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243832    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-run-systemd\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243856    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243892    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243961    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.243992    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244051    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244079    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244105    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244130    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244194    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244219    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244245    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8fh2\" (UniqueName: \"kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244271    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244296    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244298    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244320    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244494    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244546    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244569    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fa94833-b283-4190-b623-f36b7c9edf2f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244574    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244601    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244627    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244658    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244677    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244696    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244771    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244809    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244841    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244868    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244896    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244925    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244952    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.244980    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w665x\" (UniqueName: \"kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245003    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.244988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245007    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245202    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245414    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245509    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245757    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn2qz\" (UniqueName: \"kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245793    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245825    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245854    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58sqv\" (UniqueName: \"kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245895    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245951    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.245980    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246009    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246039    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246070    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246098    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246247    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246278    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5zfs\" (UniqueName: \"kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246310    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246341    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246372    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246400    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246466    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246497    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246526    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246558    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246589    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246621    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246652    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246684    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246711    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246739    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246763    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246860    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6-host-slash\") pod \"iptables-alerter-f9fz4\" (UID: \"ef54bb28-3d2f-4bc3-ae74-ded96d8dbbc6\") " pod="openshift-network-operator/iptables-alerter-f9fz4"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.246929    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-netns\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247089    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-slash\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247275    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247320    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247377    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e6a6b03c-0628-4bea-bbaa-66f0c0108dd7-sys-fs\") pod \"aws-ebs-csi-driver-node-tf5tn\" (UID: \"e6a6b03c-0628-4bea-bbaa-66f0c0108dd7\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tf5tn"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247411    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-system-cni-dir\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247580    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-metrics-server-audit-profiles\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247595    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cni-binary-copy\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247798    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.247819    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248024    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248168    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248227    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248240    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-secret-metrics-server-tls\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248298    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b78874e1-9201-4cfa-871f-1af75327111b-tmp\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248409    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248487    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-metrics-client-ca\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248632    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248659    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248714    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a209dc6c-34cd-4fd3-90f2-2591f28988a8-systemd-units\") pod \"ovnkube-node-c45gr\" (UID: \"a209dc6c-34cd-4fd3-90f2-2591f28988a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.248873    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ad08418-9a79-4912-831e-d49f5802f695-config-volume\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249068    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6f9a963b-963f-4195-9041-faa91154dc88-cnibin\") pod \"multus-additional-cni-plugins-vzxd4\" (UID: \"6f9a963b-963f-4195-9041-faa91154dc88\") " pod="openshift-multus/multus-additional-cni-plugins-vzxd4"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249270    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249600    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8e34a640-bd21-49d6-a1de-487db5d06085-original-pull-secret\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249665    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxlxg\" (UniqueName: \"kubernetes.io/projected/95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7-kube-api-access-xxlxg\") pod \"kube-state-metrics-5dc8d7d89b-b7c7n\" (UID: \"95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-b7c7n"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.249985    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/b78874e1-9201-4cfa-871f-1af75327111b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.250311    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.250399    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/068a733e-f3f0-4aec-9db1-a78b9e81c332-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.250815    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ad08418-9a79-4912-831e-d49f5802f695-metrics-tls\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251025    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-47dpt\" (UniqueName: \"kubernetes.io/projected/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-kube-api-access-47dpt\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251044    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwp8v\" (UniqueName: \"kubernetes.io/projected/81a6d809-1b40-4272-9366-466dec6031cf-kube-api-access-jwp8v\") pod \"network-metrics-daemon-ctlvw\" (UID: \"81a6d809-1b40-4272-9366-466dec6031cf\") " pod="openshift-multus/network-metrics-daemon-ctlvw"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251166    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42vnz\" (UniqueName: \"kubernetes.io/projected/d3abab98-e4d8-4ebd-b4a3-81d91079ab72-kube-api-access-42vnz\") pod \"metrics-server-8b68d8d9-5tfbm\" (UID: \"d3abab98-e4d8-4ebd-b4a3-81d91079ab72\") " pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:24.251480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251256    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8fh2\" (UniqueName: \"kubernetes.io/projected/b78874e1-9201-4cfa-871f-1af75327111b-kube-api-access-n8fh2\") pod \"thanos-querier-756469fbcb-g9zw7\" (UID: \"b78874e1-9201-4cfa-871f-1af75327111b\") " pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251535    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251535    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z67ph\" (UniqueName: \"kubernetes.io/projected/7ad08418-9a79-4912-831e-d49f5802f695-kube-api-access-z67ph\") pod \"dns-default-jmdvh\" (UID: \"7ad08418-9a79-4912-831e-d49f5802f695\") " pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.251866    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w665x\" (UniqueName: \"kubernetes.io/projected/4fa94833-b283-4190-b623-f36b7c9edf2f-kube-api-access-w665x\") pod \"monitoring-plugin-77f4f76594-72dpx\" (UID: \"4fa94833-b283-4190-b623-f36b7c9edf2f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.252021    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"prometheus-k8s-0\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.252324    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w26g\" (UniqueName: \"kubernetes.io/projected/068a733e-f3f0-4aec-9db1-a78b9e81c332-kube-api-access-8w26g\") pod \"openshift-state-metrics-7bd7489957-pm65s\" (UID: \"068a733e-f3f0-4aec-9db1-a78b9e81c332\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-pm65s"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.250374    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99jdp\" (UniqueName: \"kubernetes.io/projected/1699b579-56d9-40a2-a5c5-8253c97fd0d8-kube-api-access-99jdp\") pod \"ingress-canary-hlskz\" (UID: \"1699b579-56d9-40a2-a5c5-8253c97fd0d8\") " pod="openshift-ingress-canary/ingress-canary-hlskz"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.252490    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5zfs\" (UniqueName: \"kubernetes.io/projected/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-kube-api-access-d5zfs\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.253695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.252718    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"alertmanager-main-0\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:24.255083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.255060    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn2qz\" (UniqueName: \"kubernetes.io/projected/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-kube-api-access-kn2qz\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.255466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.255420    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58sqv\" (UniqueName: \"kubernetes.io/projected/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-kube-api-access-58sqv\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.347088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347063    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347092    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347107    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347122    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347145    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347180    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-multus-certs\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347196    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-netns\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347196    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-kubernetes\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347216    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347242    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347217    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-host\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347259    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-modprobe-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347265    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347303    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-node-exporter-wtmp\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347307    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-sys\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347322    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347334    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-lib-modules\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347342    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347363    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347389    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347409    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-bin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347413    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347419    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-sys\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347455    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c-crio-socket\") pod \"insights-runtime-extractor-v6xnw\" (UID: \"9e0be2b4-c6f4-4366-9d4b-a9b42f23c19c\") " pod="openshift-insights/insights-runtime-extractor-v6xnw"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347472    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347481    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-cnibin\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347499    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347527    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347548    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347558    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-system-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347568    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-systemd\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347574    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347583    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-conf\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.347578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347588    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-run\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347601    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347621    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347634    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-run-k8s-cni-cncf-io\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347633    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-cni-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347636    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347670    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-etc-kubernetes\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347676    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-cni-multus\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347687    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347679    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-conf-dir\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347703    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347720    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347735    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347750    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347765    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347777    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-kubelet-config\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347780    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8e34a640-bd21-49d6-a1de-487db5d06085-dbus\") pod \"global-pull-secret-syncer-26bcq\" (UID: \"8e34a640-bd21-49d6-a1de-487db5d06085\") " pod="kube-system/global-pull-secret-syncer-26bcq"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347786    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347793    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f-root\") pod \"node-exporter-srnnf\" (UID: \"2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f\") " pod="openshift-monitoring/node-exporter-srnnf"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347812    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-var-lib-kubelet\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347819    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-host-var-lib-kubelet\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347815    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysconfig\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347831    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-multus-socket-dir-parent\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347832    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347856    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347861    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e86b21ac-fc11-4f42-a3de-db0bda5e47f6-hostroot\") pod \"multus-2zxxc\" (UID: \"e86b21ac-fc11-4f42-a3de-db0bda5e47f6\") " pod="openshift-multus/multus-2zxxc"
Oct 30 21:02:24.348357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:24.347961    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc-etc-sysctl-d\") pod \"tuned-pkswx\" (UID: \"cb36fe40-e2d8-41ef-a19b-74f2e7c69cbc\") " pod="openshift-cluster-node-tuning-operator/tuned-pkswx"
Oct 30 21:02:26.561584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:26.561558    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-756469fbcb-g9zw7"
Oct 30 21:02:27.135207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:27.135175    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jmdvh"
Oct 30 21:02:28.008027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:28.008003    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:28.178958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:28.178923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:29.415643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:29.415603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:29.519655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:29.519620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:30.411973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:30.411940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:30.516304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:30.516274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:30.926601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:30.926574    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:31.327656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:31.327591    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:31.327817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:31.327806    6738 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 21:02:33.037763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:33.037736    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-72dpx"
Oct 30 21:02:33.095137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:33.095113    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-8b68d8d9-5tfbm"
Oct 30 21:02:33.993520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:33.993478    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:02:37.222019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:37.221945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:37.337196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:37.337160    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:38.237162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237127    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:38.237648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237619    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="alertmanager" containerID="cri-o://b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5" gracePeriod=120
Oct 30 21:02:38.237718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237685    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="config-reloader" containerID="cri-o://1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42" gracePeriod=120
Oct 30 21:02:38.237718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237709    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy" containerID="cri-o://03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9" gracePeriod=120
Oct 30 21:02:38.237821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237628    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-metric" containerID="cri-o://279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d" gracePeriod=120
Oct 30 21:02:38.237821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.237685    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-web" containerID="cri-o://f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171" gracePeriod=120
Oct 30 21:02:38.238242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:38.238192    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="prom-label-proxy" containerID="cri-o://47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817" gracePeriod=120
Oct 30 21:02:39.284774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284741    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817" exitCode=0
Oct 30 21:02:39.284774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284767    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9" exitCode=0
Oct 30 21:02:39.284774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284773    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42" exitCode=0
Oct 30 21:02:39.284774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284778    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5" exitCode=0
Oct 30 21:02:39.285258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284802    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"}
Oct 30 21:02:39.285258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284856    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"}
Oct 30 21:02:39.285258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284881    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"}
Oct 30 21:02:39.285258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.284906    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"}
Oct 30 21:02:39.485290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.485268    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:39.581015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.580942    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.580980    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581001    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581029    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581058    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581184    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581246    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581306    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581336    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581369    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581409    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581512    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581550    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") pod \"f666bd4a-175a-42dd-adb8-11aaf725c174\" (UID: \"f666bd4a-175a-42dd-adb8-11aaf725c174\") "
Oct 30 21:02:39.581701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581622    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:39.581844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.581824    6738 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.582583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.582113    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:02:39.582930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.582908    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:39.584472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.584401    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.584692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.584665    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "kube-api-access-7rphv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:02:39.584805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.584764    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.584922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.584895    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:02:39.585312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.585285    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.585603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.585585    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.586603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.586576    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.586892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.586871    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:02:39.590309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.590282    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.596007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.595985    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config" pod "f666bd4a-175a-42dd-adb8-11aaf725c174" (UID: "f666bd4a-175a-42dd-adb8-11aaf725c174"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:39.682691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682642    6738 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682689    6738 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682707    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7rphv\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-kube-api-access-7rphv\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682725    6738 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-main-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682738    6738 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-config-out\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682751    6738 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-cluster-tls-config\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682765    6738 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682778    6738 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f666bd4a-175a-42dd-adb8-11aaf725c174-alertmanager-main-db\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682792    6738 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682805    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f666bd4a-175a-42dd-adb8-11aaf725c174-metrics-client-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682819    6738 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f666bd4a-175a-42dd-adb8-11aaf725c174-web-config\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:39.682940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:39.682832    6738 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f666bd4a-175a-42dd-adb8-11aaf725c174-tls-assets\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:40.292738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292702    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d" exitCode=0
Oct 30 21:02:40.292738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292734    6738 generic.go:356] "Generic (PLEG): container finished" podID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerID="f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171" exitCode=0
Oct 30 21:02:40.293200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292797    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.293200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292795    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"}
Oct 30 21:02:40.293200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292872    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"}
Oct 30 21:02:40.293200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292900    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f666bd4a-175a-42dd-adb8-11aaf725c174","Type":"ContainerDied","Data":"a7d93b2a43c77ff44f28656a315d586bc300b79a4a5ab1d138088f5a4c6a9ebb"}
Oct 30 21:02:40.293200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.292945    6738 scope.go:122] "RemoveContainer" containerID="47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"
Oct 30 21:02:40.307316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.307298    6738 scope.go:122] "RemoveContainer" containerID="279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"
Oct 30 21:02:40.316639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.316615    6738 scope.go:122] "RemoveContainer" containerID="03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"
Oct 30 21:02:40.320825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.320806    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:40.324727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.324710    6738 scope.go:122] "RemoveContainer" containerID="f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"
Oct 30 21:02:40.327356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.327337    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:40.331891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.331874    6738 scope.go:122] "RemoveContainer" containerID="1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"
Oct 30 21:02:40.338926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.338912    6738 scope.go:122] "RemoveContainer" containerID="b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"
Oct 30 21:02:40.345606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.345586    6738 scope.go:122] "RemoveContainer" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"
Oct 30 21:02:40.352423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.352287    6738 scope.go:122] "RemoveContainer" containerID="47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"
Oct 30 21:02:40.352609 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.352581    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817\": container with ID starting with 47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817 not found: ID does not exist" containerID="47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"
Oct 30 21:02:40.352661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.352619    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"} err="failed to get container status \"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817\": rpc error: code = NotFound desc = could not find container \"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817\": container with ID starting with 47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817 not found: ID does not exist"
Oct 30 21:02:40.352661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.352651    6738 scope.go:122] "RemoveContainer" containerID="279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"
Oct 30 21:02:40.352895 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.352877    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d\": container with ID starting with 279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d not found: ID does not exist" containerID="279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"
Oct 30 21:02:40.352954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.352906    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"} err="failed to get container status \"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d\": rpc error: code = NotFound desc = could not find container \"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d\": container with ID starting with 279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d not found: ID does not exist"
Oct 30 21:02:40.352954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.352930    6738 scope.go:122] "RemoveContainer" containerID="03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"
Oct 30 21:02:40.353149 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.353129    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9\": container with ID starting with 03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9 not found: ID does not exist" containerID="03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"
Oct 30 21:02:40.353188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353155    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"} err="failed to get container status \"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9\": rpc error: code = NotFound desc = could not find container \"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9\": container with ID starting with 03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9 not found: ID does not exist"
Oct 30 21:02:40.353188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353168    6738 scope.go:122] "RemoveContainer" containerID="f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"
Oct 30 21:02:40.353380 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.353365    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171\": container with ID starting with f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171 not found: ID does not exist" containerID="f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"
Oct 30 21:02:40.353468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353389    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"} err="failed to get container status \"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171\": rpc error: code = NotFound desc = could not find container \"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171\": container with ID starting with f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171 not found: ID does not exist"
Oct 30 21:02:40.353468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353414    6738 scope.go:122] "RemoveContainer" containerID="1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"
Oct 30 21:02:40.353974 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.353957    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42\": container with ID starting with 1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42 not found: ID does not exist" containerID="1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"
Oct 30 21:02:40.354027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353976    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"} err="failed to get container status \"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42\": rpc error: code = NotFound desc = could not find container \"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42\": container with ID starting with 1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42 not found: ID does not exist"
Oct 30 21:02:40.354027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.353989    6738 scope.go:122] "RemoveContainer" containerID="b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"
Oct 30 21:02:40.354208 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.354192    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5\": container with ID starting with b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5 not found: ID does not exist" containerID="b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"
Oct 30 21:02:40.354249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354213    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"} err="failed to get container status \"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5\": rpc error: code = NotFound desc = could not find container \"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5\": container with ID starting with b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5 not found: ID does not exist"
Oct 30 21:02:40.354249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354228    6738 scope.go:122] "RemoveContainer" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"
Oct 30 21:02:40.354522 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:40.354504    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a\": container with ID starting with 0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a not found: ID does not exist" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"
Oct 30 21:02:40.354619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354530    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"} err="failed to get container status \"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a\": rpc error: code = NotFound desc = could not find container \"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a\": container with ID starting with 0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a not found: ID does not exist"
Oct 30 21:02:40.354619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354560    6738 scope.go:122] "RemoveContainer" containerID="47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"
Oct 30 21:02:40.355468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354954    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817"} err="failed to get container status \"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817\": rpc error: code = NotFound desc = could not find container \"47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817\": container with ID starting with 47bd1695ca02704e2defbc269a49b403cb95dd26891c39ae653c5dfd537c3817 not found: ID does not exist"
Oct 30 21:02:40.355468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.354977    6738 scope.go:122] "RemoveContainer" containerID="279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"
Oct 30 21:02:40.355468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355210    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d"} err="failed to get container status \"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d\": rpc error: code = NotFound desc = could not find container \"279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d\": container with ID starting with 279542634c52fd4024a401357d788addb7962c8723beca0225e83e1ddadc459d not found: ID does not exist"
Oct 30 21:02:40.355468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355229    6738 scope.go:122] "RemoveContainer" containerID="03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"
Oct 30 21:02:40.355653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355520    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9"} err="failed to get container status \"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9\": rpc error: code = NotFound desc = could not find container \"03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9\": container with ID starting with 03dafc23a89a6b368fddf1a06bc879c9d4e8b4c41cd74c335544d374388604e9 not found: ID does not exist"
Oct 30 21:02:40.355653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355545    6738 scope.go:122] "RemoveContainer" containerID="f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"
Oct 30 21:02:40.355813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355788    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171"} err="failed to get container status \"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171\": rpc error: code = NotFound desc = could not find container \"f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171\": container with ID starting with f32e6b3b596ba1daa4be3950033a6de03baaf6fbe433015418ae3181f4efd171 not found: ID does not exist"
Oct 30 21:02:40.355858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.355815    6738 scope.go:122] "RemoveContainer" containerID="1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"
Oct 30 21:02:40.356053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356033    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42"} err="failed to get container status \"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42\": rpc error: code = NotFound desc = could not find container \"1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42\": container with ID starting with 1b0ecc870077ba2ab0cef1ca6dc69e63ff0d7c759f2c6688ab30dc2012245a42 not found: ID does not exist"
Oct 30 21:02:40.356053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356053    6738 scope.go:122] "RemoveContainer" containerID="b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"
Oct 30 21:02:40.356179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356099    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:40.356285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356259    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5"} err="failed to get container status \"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5\": rpc error: code = NotFound desc = could not find container \"b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5\": container with ID starting with b647d5796684ef6519677c00aa39b3117461b4026e70142c9770a2290f8b5ab5 not found: ID does not exist"
Oct 30 21:02:40.356347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356286    6738 scope.go:122] "RemoveContainer" containerID="0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356415    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-web"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356458    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-web"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356485    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="prom-label-proxy"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356491    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="prom-label-proxy"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356503    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-metric"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356499    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a"} err="failed to get container status \"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a\": rpc error: code = NotFound desc = could not find container \"0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a\": container with ID starting with 0511cbe049f5b326c9121a10ac93b5c9811fcd9e5213c209bd17d321657e290a not found: ID does not exist"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356508    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-metric"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356518    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy"
Oct 30 21:02:40.356528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356522    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356536    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="init-config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356541    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="init-config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356548    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="alertmanager"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356552    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="alertmanager"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356559    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356563    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356611    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-metric"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356620    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356627    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="init-config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356633    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="alertmanager"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356639    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="config-reloader"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356644    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="kube-rbac-proxy-web"
Oct 30 21:02:40.356790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.356651    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" containerName="prom-label-proxy"
Oct 30 21:02:40.359958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.359941    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.368191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368173    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 21:02:40.368281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368177    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 21:02:40.368321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368305    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 21:02:40.368358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368323    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 21:02:40.368358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368337    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 21:02:40.368358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368350    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 21:02:40.368479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368342    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 21:02:40.368479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368459    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 21:02:40.368575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.368539    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lgbjz\""
Oct 30 21:02:40.376179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.372279    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 21:02:40.376981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.376959    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:40.488692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488657    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-web-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.488857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488700    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.488857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488726    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.488857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488825    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-out\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488908    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488945    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d466r\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-kube-api-access-d466r\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.488977    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489003    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489081    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489112    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489127    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489209    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.489247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.489227    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.589998    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590032    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590064    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-out\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590092    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590114    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d466r\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-kube-api-access-d466r\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590137    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590157    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590192    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590296    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590350    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590376    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.590726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590401    6738 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 21:02:40.590726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.590430    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-web-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.591129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.591066    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.591569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.591506    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.591569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.591555    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3fd7b2f8-1d04-4b58-9277-983f00415c56-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.593712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.593680    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.593848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.593823    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.593909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.593857    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-volume\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.594252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.594230    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-web-config\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.594333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.594258    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.594333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.594276    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.594427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.594353    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3fd7b2f8-1d04-4b58-9277-983f00415c56-config-out\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.594650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.594630    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3fd7b2f8-1d04-4b58-9277-983f00415c56-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.595498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.595481    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.598883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.598861    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d466r\" (UniqueName: \"kubernetes.io/projected/3fd7b2f8-1d04-4b58-9277-983f00415c56-kube-api-access-d466r\") pod \"alertmanager-main-0\" (UID: \"3fd7b2f8-1d04-4b58-9277-983f00415c56\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.679631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.679600    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 21:02:40.956182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:40.956123    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 21:02:40.960206 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:40.960166    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fd7b2f8_1d04_4b58_9277_983f00415c56.slice/crio-b69edb697e8f65d9dba68efece2d0fe456fb7db414899f83dcf35ed3c2158fad WatchSource:0}: Error finding container b69edb697e8f65d9dba68efece2d0fe456fb7db414899f83dcf35ed3c2158fad: Status 404 returned error can't find the container with id b69edb697e8f65d9dba68efece2d0fe456fb7db414899f83dcf35ed3c2158fad
Oct 30 21:02:41.161189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:41.161158    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f666bd4a-175a-42dd-adb8-11aaf725c174" path="/var/lib/kubelet/pods/f666bd4a-175a-42dd-adb8-11aaf725c174/volumes"
Oct 30 21:02:41.300191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:41.300112    6738 generic.go:356] "Generic (PLEG): container finished" podID="3fd7b2f8-1d04-4b58-9277-983f00415c56" containerID="48ecc63d33229f61f34b46febd9ffc6aaf5daf9671e670f52c58c1029391168c" exitCode=0
Oct 30 21:02:41.300191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:41.300160    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerDied","Data":"48ecc63d33229f61f34b46febd9ffc6aaf5daf9671e670f52c58c1029391168c"}
Oct 30 21:02:41.300696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:41.300214    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"b69edb697e8f65d9dba68efece2d0fe456fb7db414899f83dcf35ed3c2158fad"}
Oct 30 21:02:42.271519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.271480    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"]
Oct 30 21:02:42.274717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.274679    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.283033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.283004    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 21:02:42.286361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.286335    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 21:02:42.288141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.288118    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"]
Oct 30 21:02:42.307920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.307889    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"5dc79ce0880f7a3b85d7778afafea00c62ef9adc34b634e6536031c852ef0370"}
Oct 30 21:02:42.308324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.307961    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"2b81c6adfd89705831ead291d466f03ee7111a01103328bc74f90da24d30f3b9"}
Oct 30 21:02:42.308324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.308014    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"bfb71a2e6ceec429db14ac5fde54ec22452cc1ce1a32ff8f00f823aa03b5c6ba"}
Oct 30 21:02:42.308324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.308057    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"9cc1a289819a045f8c8ea7da4608ec1168d98e43cab2693d575bbd79d65e08a6"}
Oct 30 21:02:42.308324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.308101    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"8f09d96bf672af46312bdbfb719b85a2ea660240cb79a567af316ec83724cb6b"}
Oct 30 21:02:42.308324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.308141    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3fd7b2f8-1d04-4b58-9277-983f00415c56","Type":"ContainerStarted","Data":"291ff3bdab2aa82c70b7d095a0d7f12550a71292365dfbfb282d92bba9880d6e"}
Oct 30 21:02:42.343879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.343800    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.333230794 podStartE2EDuration="2.343783805s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.553011ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:42.343488696 +0000 UTC m=+19.299175926" watchObservedRunningTime="2026-10-30 21:02:42.343783805 +0000 UTC m=+19.299471033"
Oct 30 21:02:42.370168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.370141    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 21:02:42.370358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.370336    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-tw46g\""
Oct 30 21:02:42.370358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.370344    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 21:02:42.370358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.370334    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 21:02:42.370539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.370337    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 21:02:42.405574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405543    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-serving-certs-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.405574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405580    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-federate-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.405819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405707    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.405819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405752    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.405908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405862    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-metrics-client-ca\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.406019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.405993    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.406184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.406163    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl4nx\" (UniqueName: \"kubernetes.io/projected/dc1286b6-f07e-496f-8f52-475379c5b185-kube-api-access-wl4nx\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.406286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.406267    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.506869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.506834    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-serving-certs-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.506881    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-federate-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.506925    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.506956    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507000    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-metrics-client-ca\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507058    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507422    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl4nx\" (UniqueName: \"kubernetes.io/projected/dc1286b6-f07e-496f-8f52-475379c5b185-kube-api-access-wl4nx\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507504    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507672    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-serving-certs-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.507845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.507741    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-metrics-client-ca\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.508791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.508761    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.509883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.509855    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-federate-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.510021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.510003    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-telemeter-client-tls\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.510144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.510127    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.510230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.510209    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dc1286b6-f07e-496f-8f52-475379c5b185-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.515176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.515155    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl4nx\" (UniqueName: \"kubernetes.io/projected/dc1286b6-f07e-496f-8f52-475379c5b185-kube-api-access-wl4nx\") pod \"telemeter-client-5bff7c9d65-54l6l\" (UID: \"dc1286b6-f07e-496f-8f52-475379c5b185\") " pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.590376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.590300    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"
Oct 30 21:02:42.654457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654411    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:42.654831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654802    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="prometheus" containerID="cri-o://d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39" gracePeriod=600
Oct 30 21:02:42.654898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654861    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="thanos-sidecar" containerID="cri-o://9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90" gracePeriod=600
Oct 30 21:02:42.654947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654889    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-thanos" containerID="cri-o://5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861" gracePeriod=600
Oct 30 21:02:42.654947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654865    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="config-reloader" containerID="cri-o://4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be" gracePeriod=600
Oct 30 21:02:42.655045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654860    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy" containerID="cri-o://01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51" gracePeriod=600
Oct 30 21:02:42.655045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.654996    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-web" containerID="cri-o://1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad" gracePeriod=600
Oct 30 21:02:42.866356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.866331    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5bff7c9d65-54l6l"]
Oct 30 21:02:42.868388 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:02:42.868365    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1286b6_f07e_496f_8f52_475379c5b185.slice/crio-99d660929dcc02e517ccf75e119d6b29598f724e601a3419e0f3dc72e5e63712 WatchSource:0}: Error finding container 99d660929dcc02e517ccf75e119d6b29598f724e601a3419e0f3dc72e5e63712: Status 404 returned error can't find the container with id 99d660929dcc02e517ccf75e119d6b29598f724e601a3419e0f3dc72e5e63712
Oct 30 21:02:42.870389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.870373    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:02:42.891632 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:42.891612    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.011199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011159    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011202    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011228    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011262    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011387    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011458    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011500    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011533    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011541    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011563    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.011629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011612    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011650    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011695    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011735    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011770    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011807    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011866    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.011896    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.012044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012070    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012112    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.012525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012138    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012174    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012230    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") pod \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\" (UID: \"4acaeddc-b0dd-451a-8cbe-353b0a0823af\") "
Oct 30 21:02:43.012730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012532    6738 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.012730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012552    6738 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.012730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.012567    6738 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.013196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.013154    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:02:43.013642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.013618    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.014216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.013972    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.014216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.014000    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.014980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.014619    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:02:43.015317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.015291    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.015862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.015835    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.015973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.015885    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:02:43.015973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.015891    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.015973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.015950    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:02:43.016145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.016073    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.016145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.016085    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "kube-api-access-9vgkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:02:43.016356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.016334    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.016528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.016511    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.016827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.016808    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.017028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.017013    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.027863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.027840    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config" pod "4acaeddc-b0dd-451a-8cbe-353b0a0823af" (UID: "4acaeddc-b0dd-451a-8cbe-353b0a0823af"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:02:43.113656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113603    6738 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113653    6738 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-web-config\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113665    6738 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113674    6738 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113685    6738 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-grpc-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113693    6738 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config-out\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113701    6738 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-tls-assets\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113710    6738 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-metrics-client-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113718    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vgkk\" (UniqueName: \"kubernetes.io/projected/4acaeddc-b0dd-451a-8cbe-353b0a0823af-kube-api-access-9vgkk\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113726    6738 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-db\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113735    6738 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-config\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113748    6738 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113757    6738 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-kube-rbac-proxy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113771    6738 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113781    6738 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113790    6738 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4acaeddc-b0dd-451a-8cbe-353b0a0823af-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.113905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.113799    6738 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4acaeddc-b0dd-451a-8cbe-353b0a0823af-secret-metrics-client-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:02:43.315583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315551    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861" exitCode=0
Oct 30 21:02:43.315583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315581    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51" exitCode=0
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315592    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad" exitCode=0
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315600    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90" exitCode=0
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315607    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be" exitCode=0
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315614    6738 generic.go:356] "Generic (PLEG): container finished" podID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39" exitCode=0
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315633    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315659    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315710    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315729    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315748    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315781    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315816    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315852    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"}
Oct 30 21:02:43.316063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.315891    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4acaeddc-b0dd-451a-8cbe-353b0a0823af","Type":"ContainerDied","Data":"640438d0e28af12daaa161aa961a81ff3fe44c3935ea22b6eac24c0505b80f1f"}
Oct 30 21:02:43.317214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.317178    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l" event={"ID":"dc1286b6-f07e-496f-8f52-475379c5b185","Type":"ContainerStarted","Data":"99d660929dcc02e517ccf75e119d6b29598f724e601a3419e0f3dc72e5e63712"}
Oct 30 21:02:43.324206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.324190    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.331910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.331895    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.335731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.335689    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:43.339782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.339754    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:43.341482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.341466    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.349009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.348988    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.358164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.358147    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.364524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.364502    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:43.364959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.364939    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="prometheus"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.364964    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="prometheus"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.364982    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-web"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.364990    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-web"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365006    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="init-config-reloader"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365013    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="init-config-reloader"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365027    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365034    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy"
Oct 30 21:02:43.365043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365045    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-thanos"
Oct 30 21:02:43.365297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365052    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-thanos"
Oct 30 21:02:43.365297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365064    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="thanos-sidecar"
Oct 30 21:02:43.365297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365071    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="thanos-sidecar"
Oct 30 21:02:43.365297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365085    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="config-reloader"
Oct 30 21:02:43.365297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365092    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="config-reloader"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365301    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365320    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-thanos"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365330    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="thanos-sidecar"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365341    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="config-reloader"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365355    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="prometheus"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365366    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="kube-rbac-proxy-web"
Oct 30 21:02:43.365478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.365378    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" containerName="init-config-reloader"
Oct 30 21:02:43.368235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.368215    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.370258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.370236    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.375060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.374932    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 21:02:43.375206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375181    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 21:02:43.375206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375118    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 21:02:43.375521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375168    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 21:02:43.375521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375052    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 21:02:43.375521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375362    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 21:02:43.375521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375415    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 21:02:43.375735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375580    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 21:02:43.375735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375632    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 21:02:43.375735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375673    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 21:02:43.375735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.375716    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-9ljfc\""
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.376828    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7dnlj6d6sts0u\""
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.377040    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.377414    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.378171    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.378472    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.378510    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.378531    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.378811    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.378843    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.378869    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.379320    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.379346    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.379551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.379364    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.380646    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.381297    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381340    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381366    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381397    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.381640    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381667    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381688    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.381956    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.381985    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.382004    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:43.382296    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.382323    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.382344    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.382842    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.382991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.382864    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.383756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.383144    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.383756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.383180    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.383756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.383571    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.383756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.383649    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.384239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.384173    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.384393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.384301    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.384620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.384598    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.384688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.384622    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.385038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385014    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.385106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385041    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.385345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385310    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.385345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385330    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.385638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385613    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.385696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385639    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.386010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.385989    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.386050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.386011    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.386401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.386377    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.386513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.386403    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.386731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.386710    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.386775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.386732    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.387058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.387028    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.387164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.387050    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.388550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.388374    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.388550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.388399    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.389093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.389072    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.389093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.389092    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.389343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.389319    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.389343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.389344    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.390019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.389993    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.390019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390019    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.390307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390283    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.390307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390307    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.390594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390561    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.390594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390588    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.390957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390633    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:43.390957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390835    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.390957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.390848    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.391185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.391101    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.391185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.391122    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.392580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.392553    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.392699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.392580    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.392861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.392840    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.392921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.392862    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.393160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393132    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.393255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393161    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.393487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393461    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.393487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393484    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.393726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393701    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.393797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393727    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.393964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393945    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.394095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.393965    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.394342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394315    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.394415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394343    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.394614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394592    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.394692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394617    6738 scope.go:122] "RemoveContainer" containerID="5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"
Oct 30 21:02:43.394967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394941    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861"} err="failed to get container status \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": rpc error: code = NotFound desc = could not find container \"5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861\": container with ID starting with 5e22f3c40e7bf04cb9e4730ff0fe4d400453716da40d7a7c8f5c8ae4230bd861 not found: ID does not exist"
Oct 30 21:02:43.394967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.394967    6738 scope.go:122] "RemoveContainer" containerID="01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"
Oct 30 21:02:43.395394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.395373    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51"} err="failed to get container status \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": rpc error: code = NotFound desc = could not find container \"01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51\": container with ID starting with 01d12d39d9ae3bde8b7a2542b47881d08aaec374c606b37fa65e4610d2a60a51 not found: ID does not exist"
Oct 30 21:02:43.395394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.395394    6738 scope.go:122] "RemoveContainer" containerID="1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"
Oct 30 21:02:43.395785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.395758    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad"} err="failed to get container status \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": rpc error: code = NotFound desc = could not find container \"1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad\": container with ID starting with 1141d9a3cce02ccdeec78c41aba10cb40b1cbd2aa1606f02b173df0410a602ad not found: ID does not exist"
Oct 30 21:02:43.395785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.395784    6738 scope.go:122] "RemoveContainer" containerID="9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"
Oct 30 21:02:43.396143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396123    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90"} err="failed to get container status \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": rpc error: code = NotFound desc = could not find container \"9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90\": container with ID starting with 9f7216b1c8a95a1a4596f448ba86cee6f3e8c4350b5ab4166dd4cd387f45bd90 not found: ID does not exist"
Oct 30 21:02:43.396143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396144    6738 scope.go:122] "RemoveContainer" containerID="4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"
Oct 30 21:02:43.396382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396359    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be"} err="failed to get container status \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": rpc error: code = NotFound desc = could not find container \"4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be\": container with ID starting with 4b5687b29cc584d1ca180e7780eaa8d8a92c7c30575b2ac14bc6e571678b85be not found: ID does not exist"
Oct 30 21:02:43.396382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396376    6738 scope.go:122] "RemoveContainer" containerID="d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"
Oct 30 21:02:43.396645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396624    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39"} err="failed to get container status \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": rpc error: code = NotFound desc = could not find container \"d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39\": container with ID starting with d6bcccdf11c0c1edf31ce2ead09625be19affa57429830bc4f8e67b529b89d39 not found: ID does not exist"
Oct 30 21:02:43.396727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396647    6738 scope.go:122] "RemoveContainer" containerID="ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"
Oct 30 21:02:43.396976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.396823    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e"} err="failed to get container status \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": rpc error: code = NotFound desc = could not find container \"ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e\": container with ID starting with ef4f4a2d2bdfa8ded4f371e3e97c6a14a26b916a8285df397b6555867945dc9e not found: ID does not exist"
Oct 30 21:02:43.516191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516120    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516160    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516196    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516233    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4qph\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-kube-api-access-h4qph\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516347    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516373    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516421    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516511    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516540    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516566    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516584    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516609    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516625    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516644    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516664    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516689    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516713    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516740    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516760    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.516818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.516781    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.617692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617653    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.617860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617713    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.617860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617775    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.617860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617805    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.617860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617838    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617864    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617891    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617932    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617959    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.617983    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618013    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618049    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618082    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618107    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4qph\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-kube-api-access-h4qph\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618148    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618172    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618195    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618226    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618249    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618281    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618686    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618931    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.618984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.618686    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.621535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.620024    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.621535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.620682    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.621535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.621213    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.621535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.621348    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.622008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.621980    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.622156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.622131    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.623420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.623394    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.624076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.624049    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.624415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.624393    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.624938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.624914    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.625196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.625172    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.625271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.625206    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.625802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.625722    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.626166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.626144    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.626935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.626909    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-web-config\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.627274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.627250    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-config-out\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.627414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.627398    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4qph\" (UniqueName: \"kubernetes.io/projected/e7861629-db0d-4d8b-aa64-f4ba432dd2cd-kube-api-access-h4qph\") pod \"prometheus-k8s-0\" (UID: \"e7861629-db0d-4d8b-aa64-f4ba432dd2cd\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.697508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.697472    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:43.991357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:43.991254    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 21:02:44.322467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:44.322376    6738 generic.go:356] "Generic (PLEG): container finished" podID="e7861629-db0d-4d8b-aa64-f4ba432dd2cd" containerID="944b9bf651764e11ff98554527b68b1a9f9a5fa6f195455a6f42a9fdcd68bf9f" exitCode=0
Oct 30 21:02:44.322890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:44.322473    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerDied","Data":"944b9bf651764e11ff98554527b68b1a9f9a5fa6f195455a6f42a9fdcd68bf9f"}
Oct 30 21:02:44.322890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:44.322552    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"8fa2aa3a65789b7077811431d2f6159c43cf17c828b949d61dd9e9a5d9bd7501"}
Oct 30 21:02:45.161119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.161082    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4acaeddc-b0dd-451a-8cbe-353b0a0823af" path="/var/lib/kubelet/pods/4acaeddc-b0dd-451a-8cbe-353b0a0823af/volumes"
Oct 30 21:02:45.331057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331018    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"37418c449b3e7e8bd839316c8e8befdb153687694f4ca139c41095c2afd2f8f0"}
Oct 30 21:02:45.331470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331091    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"fb4224e71d54b91e55376be8c7c8d74c6c99bbe691479654f3b75e0820be97ac"}
Oct 30 21:02:45.331470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331139    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"56ab3e3db7cfd879cea3dab370995a21979da5de42f395992f4681d0684bd091"}
Oct 30 21:02:45.331470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331171    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"f28aea1225d7d782c443163879e912831eb84843b01bc37f589d77c5975f0cf6"}
Oct 30 21:02:45.331470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331199    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"9452e34b11e159b9ed13d25cbcb6bc237cc3dfa44d5db29deee4d39a07e47c26"}
Oct 30 21:02:45.331470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.331237    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e7861629-db0d-4d8b-aa64-f4ba432dd2cd","Type":"ContainerStarted","Data":"8e4bd39eab7bcad87809703ef7769e1b21b13525d56af70a70d79977803d4e88"}
Oct 30 21:02:45.333117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.333089    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l" event={"ID":"dc1286b6-f07e-496f-8f52-475379c5b185","Type":"ContainerStarted","Data":"fe791f2ce0e26e7e78679fce06acd2a6aaaa6968d36056dabcfcd3d1767fe9b9"}
Oct 30 21:02:45.333213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.333144    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l" event={"ID":"dc1286b6-f07e-496f-8f52-475379c5b185","Type":"ContainerStarted","Data":"ea6c613a08db1dfab89cab2fdb88ecc5f3e5228f7bc2d2778a3b543b9d6728b7"}
Oct 30 21:02:45.333213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.333173    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l" event={"ID":"dc1286b6-f07e-496f-8f52-475379c5b185","Type":"ContainerStarted","Data":"c9530926f1e331edf8c4960637489849c8223ef40c23f3d51d1e10288f591d45"}
Oct 30 21:02:45.362619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.361679    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.341903735 podStartE2EDuration="2.361666999s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.763264ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:45.360385856 +0000 UTC m=+22.316073085" watchObservedRunningTime="2026-10-30 21:02:45.361666999 +0000 UTC m=+22.317354217"
Oct 30 21:02:45.384895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.384847    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5bff7c9d65-54l6l" podStartSLOduration=1.832709395 podStartE2EDuration="3.384831685s" totalImagesPullingTime="1.55212229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:02:45.383125524 +0000 UTC m=+22.338812748" watchObservedRunningTime="2026-10-30 21:02:45.384831685 +0000 UTC m=+22.340518905"
Oct 30 21:02:45.575246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.575176    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:45.693772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.693747    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:45.944148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:45.944121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:46.004831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:46.004802    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-c45gr"
Oct 30 21:02:46.059806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:46.059774    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:48.117032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:48.117002    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-m474j"
Oct 30 21:02:48.698532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:48.698503    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:02:49.178243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:49.178211    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:49.282287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:49.282258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:49.385486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:49.385427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:02:52.506666 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:02:52.506630    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:02:53.996254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:02:53.996224    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nkrzz"
Oct 30 21:03:01.135593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:01.135560    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:03:01.151977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:01.151954    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:03:01.412773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:01.412694    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 21:03:02.547583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:02.547552    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:02.685678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:02.685643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:04.773538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:04.773503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:04.896070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:04.896035    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:05.086197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:05.086133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:05.196474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:05.196427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:16.796334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:16.796294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:18.842029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:18.841984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:18.946615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:18.946567    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:22.360834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:22.360795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:22.465944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:03:22.465911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:03:22.507769 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:03:22.507730    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:03:52.506555 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:03:52.506521    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:04:13.924943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:04:13.924903    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:16.833114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:04:16.833082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:04:22.509577 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:04:22.509534    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:04:52.509275 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:04:52.509230    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:05:16.902801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:16.902756    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.545691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:17.545644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:17.686002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:17.685966    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:20.857395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:20.857354    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:21.176950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:21.176915    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:21.406678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:21.406641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:22.506386 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:05:22.506353    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:05:23.427264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:23.427237    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:23.699955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:23.699778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:25.218645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:25.218609    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:27.417630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:27.417593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:27.560055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:27.560015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:28.426534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:28.426480    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:28.566301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:28.566259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:29.001315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.001275    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"]
Oct 30 21:05:29.004893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.004873    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.013749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.013728    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-c2fg9\""
Oct 30 21:05:29.013957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.013935    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:05:29.014859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.014815    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:05:29.015629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.015608    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"]
Oct 30 21:05:29.094877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.094849    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.094976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.094900    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zlsk\" (UniqueName: \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.095018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.094981    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.196101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.196076    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.196192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.196119    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zlsk\" (UniqueName: \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.196285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.196248    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.196475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.196459    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.196532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.196520    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.205744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.205721    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zlsk\" (UniqueName: \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.323945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.323893    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:29.603630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.603608    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"]
Oct 30 21:05:29.606375 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:05:29.606345    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c63a65a_be9f_407f_9c7b_0b4b0493d520.slice/crio-6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d WatchSource:0}: Error finding container 6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d: Status 404 returned error can't find the container with id 6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d
Oct 30 21:05:29.958966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:29.958934    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4" event={"ID":"5c63a65a-be9f-407f-9c7b-0b4b0493d520","Type":"ContainerStarted","Data":"6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d"}
Oct 30 21:05:31.474802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:31.474752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:35.986448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:35.986405    6738 generic.go:356] "Generic (PLEG): container finished" podID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerID="7db29ad5fb555b756019c475836fabcdc9d60b9fd77bc1d790aa9cf3201d6aee" exitCode=0
Oct 30 21:05:35.986790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:35.986499    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4" event={"ID":"5c63a65a-be9f-407f-9c7b-0b4b0493d520","Type":"ContainerDied","Data":"7db29ad5fb555b756019c475836fabcdc9d60b9fd77bc1d790aa9cf3201d6aee"}
Oct 30 21:05:39.000335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:39.000286    6738 generic.go:356] "Generic (PLEG): container finished" podID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerID="d7dfb19dac23e4b8435776a702481164aa47795118b747ac8cbda584f4b2cb0f" exitCode=0
Oct 30 21:05:39.000886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:39.000373    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4" event={"ID":"5c63a65a-be9f-407f-9c7b-0b4b0493d520","Type":"ContainerDied","Data":"d7dfb19dac23e4b8435776a702481164aa47795118b747ac8cbda584f4b2cb0f"}
Oct 30 21:05:45.030883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:45.030841    6738 generic.go:356] "Generic (PLEG): container finished" podID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerID="64f1bc877e32525ed0e7927ddf17cc786c8cd3f7fc7d0a0249c833c7ea3b2bd2" exitCode=0
Oct 30 21:05:45.031200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:45.030936    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4" event={"ID":"5c63a65a-be9f-407f-9c7b-0b4b0493d520","Type":"ContainerDied","Data":"64f1bc877e32525ed0e7927ddf17cc786c8cd3f7fc7d0a0249c833c7ea3b2bd2"}
Oct 30 21:05:45.341005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:45.340972    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:46.158286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.158264    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:46.242821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.242793    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\" (UniqueName: \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\") pod \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") "
Oct 30 21:05:46.242959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.242847    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\") pod \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") "
Oct 30 21:05:46.242959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.242898    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\") pod \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\" (UID: \"5c63a65a-be9f-407f-9c7b-0b4b0493d520\") "
Oct 30 21:05:46.243449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.243407    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle" pod "5c63a65a-be9f-407f-9c7b-0b4b0493d520" (UID: "5c63a65a-be9f-407f-9c7b-0b4b0493d520"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:46.245209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.245188    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk" pod "5c63a65a-be9f-407f-9c7b-0b4b0493d520" (UID: "5c63a65a-be9f-407f-9c7b-0b4b0493d520"). InnerVolumeSpecName "kube-api-access-5zlsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:05:46.247850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.247832    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util" pod "5c63a65a-be9f-407f-9c7b-0b4b0493d520" (UID: "5c63a65a-be9f-407f-9c7b-0b4b0493d520"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:05:46.343904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.343854    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zlsk\" (UniqueName: \"kubernetes.io/projected/5c63a65a-be9f-407f-9c7b-0b4b0493d520-kube-api-access-5zlsk\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:46.343904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.343876    6738 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-util\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:46.343904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:46.343885    6738 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c63a65a-be9f-407f-9c7b-0b4b0493d520-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:05:47.040820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:47.040780    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4" event={"ID":"5c63a65a-be9f-407f-9c7b-0b4b0493d520","Type":"ContainerDied","Data":"6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d"}
Oct 30 21:05:47.040820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:47.040815    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644792w4"
Oct 30 21:05:47.041022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:47.040833    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6b5e605f071c710f240ee0ddc92d5f926c636f192490591dcf14d72b3b78005d"
Oct 30 21:05:49.360615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:49.360586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:49.467553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:49.467530    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:50.955818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.955785    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"]
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956096    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="extract"
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956106    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="extract"
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956119    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="pull"
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956123    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="pull"
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956159    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="util"
Oct 30 21:05:50.956189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956164    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="util"
Oct 30 21:05:50.956420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.956210    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c63a65a-be9f-407f-9c7b-0b4b0493d520" containerName="extract"
Oct 30 21:05:50.978348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.978327    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"]
Oct 30 21:05:50.978513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.978424    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:50.982529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.982500    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-g64p2\""
Oct 30 21:05:50.982529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.982522    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Oct 30 21:05:50.982700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:50.982585    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Oct 30 21:05:51.081363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.081343    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf72l\" (UniqueName: \"kubernetes.io/projected/f451fb0d-7206-4bd0-9d96-e286746fa12d-kube-api-access-lf72l\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.081514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.081378    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f451fb0d-7206-4bd0-9d96-e286746fa12d-tmp\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.182570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.182548    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf72l\" (UniqueName: \"kubernetes.io/projected/f451fb0d-7206-4bd0-9d96-e286746fa12d-kube-api-access-lf72l\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.182661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.182580    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f451fb0d-7206-4bd0-9d96-e286746fa12d-tmp\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.182914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.182897    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f451fb0d-7206-4bd0-9d96-e286746fa12d-tmp\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.190581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.190556    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf72l\" (UniqueName: \"kubernetes.io/projected/f451fb0d-7206-4bd0-9d96-e286746fa12d-kube-api-access-lf72l\") pod \"jobset-operator-674cd59dc5-rsk95\" (UID: \"f451fb0d-7206-4bd0-9d96-e286746fa12d\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.308856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.308788    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"
Oct 30 21:05:51.609298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.609272    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95"]
Oct 30 21:05:51.611706 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:05:51.611674    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf451fb0d_7206_4bd0_9d96_e286746fa12d.slice/crio-885088328bb433f24c9c476d90242de885ced441234a93034a7c3d39a198fe9f WatchSource:0}: Error finding container 885088328bb433f24c9c476d90242de885ced441234a93034a7c3d39a198fe9f: Status 404 returned error can't find the container with id 885088328bb433f24c9c476d90242de885ced441234a93034a7c3d39a198fe9f
Oct 30 21:05:51.832571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.832544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:51.950137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:51.950115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:52.061738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:52.061718    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95" event={"ID":"f451fb0d-7206-4bd0-9d96-e286746fa12d","Type":"ContainerStarted","Data":"885088328bb433f24c9c476d90242de885ced441234a93034a7c3d39a198fe9f"}
Oct 30 21:05:52.504900 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:05:52.504874    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:05:54.493757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:54.493708    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:54.622053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:54.622020    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:54.760016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:54.759922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:05:55.077537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:55.077476    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95" event={"ID":"f451fb0d-7206-4bd0-9d96-e286746fa12d","Type":"ContainerStarted","Data":"ca84676bd3bd36e21ebdfdf9753ca6c99763224cc1366efd49e1b61eb7f5b681"}
Oct 30 21:05:55.098147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:05:55.098018    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-rsk95" podStartSLOduration=2.115111054 podStartE2EDuration="5.098003224s" totalImagesPullingTime="2.98289217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:05:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:05:55.097595002 +0000 UTC m=+212.053282228" watchObservedRunningTime="2026-10-30 21:05:55.098003224 +0000 UTC m=+212.053690451"
Oct 30 21:06:15.837668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:15.837618    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:15.969363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:15.969325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.171728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.171691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.478373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.478307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.610917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.610892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.617556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.617535    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"]
Oct 30 21:06:16.619670 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.619656    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.657781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.657760    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"]
Oct 30 21:06:16.674112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674090    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.674207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674131    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-tmp\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.674207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674150    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdcz8\" (UniqueName: \"kubernetes.io/projected/2866e1bd-2117-4109-86f8-a79199640d5d-kube-api-access-jdcz8\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.674284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674263    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.674319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674284    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-manifests\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.674390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.674368    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.712286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712260    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 21:06:16.712286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712277    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 21:06:16.712286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712284    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 21:06:16.712532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712323    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wf7xj\""
Oct 30 21:06:16.712532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712335    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 21:06:16.712532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.712502    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 21:06:16.775430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775381    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-tmp\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775409    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdcz8\" (UniqueName: \"kubernetes.io/projected/2866e1bd-2117-4109-86f8-a79199640d5d-kube-api-access-jdcz8\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775454    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775482    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-manifests\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775627    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775690    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.775843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.775825    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-manifests\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.777968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.777942    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2866e1bd-2117-4109-86f8-a79199640d5d-tmp\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.778227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.778210    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-webhook-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.778296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.778236    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-metrics-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.778296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.778236    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2866e1bd-2117-4109-86f8-a79199640d5d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.788486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.788464    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdcz8\" (UniqueName: \"kubernetes.io/projected/2866e1bd-2117-4109-86f8-a79199640d5d-kube-api-access-jdcz8\") pod \"opendatahub-operator-controller-manager-7bd6789b8f-2txfb\" (UID: \"2866e1bd-2117-4109-86f8-a79199640d5d\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:16.818457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.818409    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.932713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.932689    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:16.933630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:16.933612    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:17.043526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:17.043465    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:17.223802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:17.223782    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"]
Oct 30 21:06:17.226142 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:17.226118    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2866e1bd_2117_4109_86f8_a79199640d5d.slice/crio-1565bbec3515ac72dc14c1327f05ba50adb5f73fcf72be3f91f772620452b341 WatchSource:0}: Error finding container 1565bbec3515ac72dc14c1327f05ba50adb5f73fcf72be3f91f772620452b341: Status 404 returned error can't find the container with id 1565bbec3515ac72dc14c1327f05ba50adb5f73fcf72be3f91f772620452b341
Oct 30 21:06:18.160721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:18.160679    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb" event={"ID":"2866e1bd-2117-4109-86f8-a79199640d5d","Type":"ContainerStarted","Data":"1565bbec3515ac72dc14c1327f05ba50adb5f73fcf72be3f91f772620452b341"}
Oct 30 21:06:19.077836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:19.077786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:19.189415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:19.189335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:21.174630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:21.174597    6738 generic.go:356] "Generic (PLEG): container finished" podID="2866e1bd-2117-4109-86f8-a79199640d5d" containerID="697dcf6e54203af870ca750c3735198723971b8199b2b38e55bab0878007c0ca" exitCode=0
Oct 30 21:06:21.175079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:21.174676    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb" event={"ID":"2866e1bd-2117-4109-86f8-a79199640d5d","Type":"ContainerDied","Data":"697dcf6e54203af870ca750c3735198723971b8199b2b38e55bab0878007c0ca"}
Oct 30 21:06:22.045361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:22.045331    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:22.180147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:22.180117    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb" event={"ID":"2866e1bd-2117-4109-86f8-a79199640d5d","Type":"ContainerStarted","Data":"9a869716e3899df5e85b8a0b9675cd6ff5faf50589a24cd10e2c4f48ddcccc1a"}
Oct 30 21:06:22.180592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:22.180225    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:22.322458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:22.322344    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb" podStartSLOduration=2.584620338 podStartE2EDuration="6.322328053s" totalImagesPullingTime="3.322386284s" totalInitContainerRuntime="415.321431ms" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:22.319462885 +0000 UTC m=+239.275150108" watchObservedRunningTime="2026-10-30 21:06:22.322328053 +0000 UTC m=+239.278015281"
Oct 30 21:06:22.508686 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:06:22.508647    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:06:23.989573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:23.989546    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:24.071745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.071704    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"]
Oct 30 21:06:24.074744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.074724    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.079659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.079629    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 21:06:24.080618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.080602    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-pqkx7\""
Oct 30 21:06:24.080674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.080615    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 21:06:24.097645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.097617    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"]
Oct 30 21:06:24.136474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.136429    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.136586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.136539    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t6gl\" (UniqueName: \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.136641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.136605    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.177294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.177269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:24.237907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.237880    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.238050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.238029    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t6gl\" (UniqueName: \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.238230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.238207    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.238313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.238263    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.238581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.238558    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.246235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.246180    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t6gl\" (UniqueName: \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.334332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.334309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:24.389331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.389312    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:24.662718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:24.662685    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"]
Oct 30 21:06:24.665554 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:24.665527    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0e8e1e4_5b68_458f_ba9e_e40131dac0ad.slice/crio-29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359 WatchSource:0}: Error finding container 29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359: Status 404 returned error can't find the container with id 29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359
Oct 30 21:06:25.192087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:25.192056    6738 generic.go:356] "Generic (PLEG): container finished" podID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerID="170c7bbcb7967a05ada066bd95c4cb0605158fca83ea48ce3dc2c19f4c3f1ca9" exitCode=0
Oct 30 21:06:25.192411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:25.192120    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr" event={"ID":"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad","Type":"ContainerDied","Data":"170c7bbcb7967a05ada066bd95c4cb0605158fca83ea48ce3dc2c19f4c3f1ca9"}
Oct 30 21:06:25.192411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:25.192169    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr" event={"ID":"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad","Type":"ContainerStarted","Data":"29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359"}
Oct 30 21:06:26.198140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:26.198078    6738 generic.go:356] "Generic (PLEG): container finished" podID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerID="3144fbe074b44ce0c0610d0c98df39e107bc38fa358f684c8e449a748aee6a08" exitCode=0
Oct 30 21:06:26.198454 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:26.198145    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr" event={"ID":"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad","Type":"ContainerDied","Data":"3144fbe074b44ce0c0610d0c98df39e107bc38fa358f684c8e449a748aee6a08"}
Oct 30 21:06:27.204001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:27.203968    6738 generic.go:356] "Generic (PLEG): container finished" podID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerID="dee285fd2e7edcd92be8c7ff75ef33f687cdd245add744db94b4c58d435604ad" exitCode=0
Oct 30 21:06:27.204384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:27.204013    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr" event={"ID":"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad","Type":"ContainerDied","Data":"dee285fd2e7edcd92be8c7ff75ef33f687cdd245add744db94b4c58d435604ad"}
Oct 30 21:06:27.286970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:27.286946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:27.458902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:27.458845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:28.328094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.328073    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:28.370927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.370896    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\") pod \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") "
Oct 30 21:06:28.371043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.370943    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\" (UniqueName: \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\") pod \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") "
Oct 30 21:06:28.371043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.370982    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\") pod \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\" (UID: \"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad\") "
Oct 30 21:06:28.371524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.371497    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle" pod "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" (UID: "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:06:28.373178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.373151    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl" pod "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" (UID: "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad"). InnerVolumeSpecName "kube-api-access-8t6gl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:06:28.378225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.378199    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util" pod "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" (UID: "b0e8e1e4-5b68-458f-ba9e-e40131dac0ad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:06:28.471819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.471748    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t6gl\" (UniqueName: \"kubernetes.io/projected/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-kube-api-access-8t6gl\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:06:28.471819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.471787    6738 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-util\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:06:28.471819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:28.471798    6738 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b0e8e1e4-5b68-458f-ba9e-e40131dac0ad-bundle\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:06:29.214031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.214007    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr"
Oct 30 21:06:29.214031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.214018    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847afhchr" event={"ID":"b0e8e1e4-5b68-458f-ba9e-e40131dac0ad","Type":"ContainerDied","Data":"29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359"}
Oct 30 21:06:29.214192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.214054    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29461357030c0469352bd9ebfbcabc34bbaa90da9b0753b5b7d9ab3a8016d359"
Oct 30 21:06:29.275320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.275300    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:29.429158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.429097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:29.590087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590062    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"]
Oct 30 21:06:29.590407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590395    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="util"
Oct 30 21:06:29.590479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590410    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="util"
Oct 30 21:06:29.590479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590421    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="pull"
Oct 30 21:06:29.590479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590445    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="pull"
Oct 30 21:06:29.590479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590472    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="extract"
Oct 30 21:06:29.590479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590477    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="extract"
Oct 30 21:06:29.590619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.590544    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0e8e1e4-5b68-458f-ba9e-e40131dac0ad" containerName="extract"
Oct 30 21:06:29.594534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.594518    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.609520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.609495    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 21:06:29.609520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.609510    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 21:06:29.610400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.610385    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fpth8\""
Oct 30 21:06:29.611305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.611285    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"]
Oct 30 21:06:29.681813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.681766    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.681919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.681831    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr4r6\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-kube-api-access-dr4r6\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.783188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.783151    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr4r6\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-kube-api-access-dr4r6\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.783384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.783206    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.795985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.795950    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.796109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.795962    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr4r6\" (UniqueName: \"kubernetes.io/projected/85acc021-6b4f-4be0-b12c-b4641b901e56-kube-api-access-dr4r6\") pod \"cert-manager-webhook-5dbbbf75cc-88wlb\" (UID: \"85acc021-6b4f-4be0-b12c-b4641b901e56\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:29.909493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:29.909470    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:30.183270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:30.183204    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"]
Oct 30 21:06:30.185385 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:30.185356    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85acc021_6b4f_4be0_b12c_b4641b901e56.slice/crio-5e9ded2a18c958885786b524b5b93120ac3ab0a8b33b628116435bbc49fb3615 WatchSource:0}: Error finding container 5e9ded2a18c958885786b524b5b93120ac3ab0a8b33b628116435bbc49fb3615: Status 404 returned error can't find the container with id 5e9ded2a18c958885786b524b5b93120ac3ab0a8b33b628116435bbc49fb3615
Oct 30 21:06:30.219641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:30.219587    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb" event={"ID":"85acc021-6b4f-4be0-b12c-b4641b901e56","Type":"ContainerStarted","Data":"5e9ded2a18c958885786b524b5b93120ac3ab0a8b33b628116435bbc49fb3615"}
Oct 30 21:06:31.538567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:31.538536    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:31.697784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:31.697703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:33.186328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:33.186303    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7bd6789b8f-2txfb"
Oct 30 21:06:33.424974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:33.424937    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:33.534180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:33.534104    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:34.240876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:34.240823    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb" event={"ID":"85acc021-6b4f-4be0-b12c-b4641b901e56","Type":"ContainerStarted","Data":"3e54f258f94b1abc0ffee2aee31ed7bbf6de800cba95a3309a9f337d5c53195f"}
Oct 30 21:06:34.241218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:34.240889    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:34.267311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:34.267268    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb" podStartSLOduration=1.854082786 podStartE2EDuration="5.267254162s" totalImagesPullingTime="3.413171376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:34.266590255 +0000 UTC m=+251.222277480" watchObservedRunningTime="2026-10-30 21:06:34.267254162 +0000 UTC m=+251.222941385"
Oct 30 21:06:35.642223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:35.642195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:35.897299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:35.897215    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:36.025031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:36.024997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:39.700915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:39.700881    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:40.209617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:40.209584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:40.247035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:40.247009    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-88wlb"
Oct 30 21:06:43.010762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:43.010725    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:43.616666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:43.616636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:43.879286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:43.879220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:44.213958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:44.213890    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:45.353777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:45.353742    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:46.255396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:46.255369    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:47.444742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.444699    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"]
Oct 30 21:06:47.448656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.448638    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.463788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.463741    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 21:06:47.464390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.464371    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 21:06:47.465009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.464978    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-2vc5j\""
Oct 30 21:06:47.466065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.466048    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"]
Oct 30 21:06:47.540395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.540352    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a19267cc-024e-466e-a236-266f5c1f5f41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.540546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.540524    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6cz9\" (UniqueName: \"kubernetes.io/projected/a19267cc-024e-466e-a236-266f5c1f5f41-kube-api-access-t6cz9\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.616653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.616624    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:47.641215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.641189    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6cz9\" (UniqueName: \"kubernetes.io/projected/a19267cc-024e-466e-a236-266f5c1f5f41-kube-api-access-t6cz9\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.641304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.641229    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a19267cc-024e-466e-a236-266f5c1f5f41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.641572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.641554    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a19267cc-024e-466e-a236-266f5c1f5f41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.649187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.649171    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6cz9\" (UniqueName: \"kubernetes.io/projected/a19267cc-024e-466e-a236-266f5c1f5f41-kube-api-access-t6cz9\") pod \"kuadrant-operator-controller-manager-678d87766f-nxr8f\" (UID: \"a19267cc-024e-466e-a236-266f5c1f5f41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.761689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.761604    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:47.841235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:47.841192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:48.020111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:48.020041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:48.102200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:48.102152    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"]
Oct 30 21:06:48.105690 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:48.105652    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda19267cc_024e_466e_a236_266f5c1f5f41.slice/crio-9329ae59931f3717ed1c8eff592cb8372a0125a389cb9a8f53871395f29195ac WatchSource:0}: Error finding container 9329ae59931f3717ed1c8eff592cb8372a0125a389cb9a8f53871395f29195ac: Status 404 returned error can't find the container with id 9329ae59931f3717ed1c8eff592cb8372a0125a389cb9a8f53871395f29195ac
Oct 30 21:06:48.243131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:48.243094    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:48.295935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:48.295858    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f" event={"ID":"a19267cc-024e-466e-a236-266f5c1f5f41","Type":"ContainerStarted","Data":"9329ae59931f3717ed1c8eff592cb8372a0125a389cb9a8f53871395f29195ac"}
Oct 30 21:06:51.450613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:51.450576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:52.512024 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:06:52.511486    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:06:52.655349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:52.655276    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:53.195483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.195455    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"]
Oct 30 21:06:53.199168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.199146    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.206020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.205999    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"]
Oct 30 21:06:53.207726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.207696    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 21:06:53.207851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.207752    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 21:06:53.208093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.208072    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vp64d\""
Oct 30 21:06:53.234412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.234384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:53.289689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.289665    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.289795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.289778    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.289839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.289811    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-285qf\" (UniqueName: \"kubernetes.io/projected/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-kube-api-access-285qf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.324497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.324470    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f" event={"ID":"a19267cc-024e-466e-a236-266f5c1f5f41","Type":"ContainerStarted","Data":"66d6e734ffb4bea43d19cf28b1544849b337a932be358bd3fbd95f1ec1e450c6"}
Oct 30 21:06:53.324602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.324580    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:06:53.343100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.343075    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:53.344151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.344109    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f" podStartSLOduration=2.072145235 podStartE2EDuration="6.344098043s" totalImagesPullingTime="4.271952808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:53.341168496 +0000 UTC m=+270.296855723" watchObservedRunningTime="2026-10-30 21:06:53.344098043 +0000 UTC m=+270.299785268"
Oct 30 21:06:53.390918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.390900    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.391008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.390928    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-285qf\" (UniqueName: \"kubernetes.io/projected/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-kube-api-access-285qf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.391008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.390955    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.391592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.391569    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.393452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.393420    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.398409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.398388    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-285qf\" (UniqueName: \"kubernetes.io/projected/0e5fb926-7759-4b04-9f9e-5c933d2c62d5-kube-api-access-285qf\") pod \"kuadrant-console-plugin-66c4fb575d-z767s\" (UID: \"0e5fb926-7759-4b04-9f9e-5c933d2c62d5\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.512302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.512243    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"
Oct 30 21:06:53.788061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.787980    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s"]
Oct 30 21:06:53.793178 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:53.793147    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e5fb926_7759_4b04_9f9e_5c933d2c62d5.slice/crio-e3f14c2dab970535f071241bff31b7f3a47544932eef0b48f9dd4419fa75b572 WatchSource:0}: Error finding container e3f14c2dab970535f071241bff31b7f3a47544932eef0b48f9dd4419fa75b572: Status 404 returned error can't find the container with id e3f14c2dab970535f071241bff31b7f3a47544932eef0b48f9dd4419fa75b572
Oct 30 21:06:53.960697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:53.960666    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.148605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.148575    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.315562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.315527    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"]
Oct 30 21:06:54.320471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.320451    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.330385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.330344    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 21:06:54.330385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.330350    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 21:06:54.333065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.333043    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s" event={"ID":"0e5fb926-7759-4b04-9f9e-5c933d2c62d5","Type":"ContainerStarted","Data":"e3f14c2dab970535f071241bff31b7f3a47544932eef0b48f9dd4419fa75b572"}
Oct 30 21:06:54.347370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.347350    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"]
Oct 30 21:06:54.357826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.357798    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 21:06:54.358203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.358117    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-t6lsf\""
Oct 30 21:06:54.358322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.358267    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 21:06:54.363244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.362602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.399283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.399230    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a025c6-fe78-42c0-849f-00afa1a76084-tls-certs\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.399283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.399272    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a025c6-fe78-42c0-849f-00afa1a76084-tmp\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.399402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.399297    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9r7\" (UniqueName: \"kubernetes.io/projected/56a025c6-fe78-42c0-849f-00afa1a76084-kube-api-access-bc9r7\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.500771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.500731    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a025c6-fe78-42c0-849f-00afa1a76084-tls-certs\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.500953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.500796    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a025c6-fe78-42c0-849f-00afa1a76084-tmp\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.500953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.500838    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9r7\" (UniqueName: \"kubernetes.io/projected/56a025c6-fe78-42c0-849f-00afa1a76084-kube-api-access-bc9r7\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.508360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.508329    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a025c6-fe78-42c0-849f-00afa1a76084-tmp\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.511959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.511933    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56a025c6-fe78-42c0-849f-00afa1a76084-tls-certs\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.512284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.512261    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9r7\" (UniqueName: \"kubernetes.io/projected/56a025c6-fe78-42c0-849f-00afa1a76084-kube-api-access-bc9r7\") pod \"kube-auth-proxy-5978974d9d-zr6ts\" (UID: \"56a025c6-fe78-42c0-849f-00afa1a76084\") " pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.538582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.538547    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.636540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.636511    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"
Oct 30 21:06:54.668261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.668177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.858109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.858071    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:54.916693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.916667    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts"]
Oct 30 21:06:54.920514 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:06:54.920420    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a025c6_fe78_42c0_849f_00afa1a76084.slice/crio-f67e3e3ccd877dad3d08035be17a4fc84f0542208224f2ab8341b25431b0afc0 WatchSource:0}: Error finding container f67e3e3ccd877dad3d08035be17a4fc84f0542208224f2ab8341b25431b0afc0: Status 404 returned error can't find the container with id f67e3e3ccd877dad3d08035be17a4fc84f0542208224f2ab8341b25431b0afc0
Oct 30 21:06:54.977247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:54.977202    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:55.345974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:55.345897    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts" event={"ID":"56a025c6-fe78-42c0-849f-00afa1a76084","Type":"ContainerStarted","Data":"f67e3e3ccd877dad3d08035be17a4fc84f0542208224f2ab8341b25431b0afc0"}
Oct 30 21:06:56.089243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:56.089152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:56.337674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:56.337618    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:58.365138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:58.365101    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts" event={"ID":"56a025c6-fe78-42c0-849f-00afa1a76084","Type":"ContainerStarted","Data":"b9bd26b75f25220b6f6e815e44deca611f30cfb5bdf0ffe5da1b9f6579f0169f"}
Oct 30 21:06:58.386360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:58.386306    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5978974d9d-zr6ts" podStartSLOduration=1.4908529449999999 podStartE2EDuration="4.386288915s" totalImagesPullingTime="2.89543597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:06:58.384362449 +0000 UTC m=+275.340049698" watchObservedRunningTime="2026-10-30 21:06:58.386288915 +0000 UTC m=+275.341976144"
Oct 30 21:06:59.017803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:59.017769    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:06:59.136578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:06:59.136521    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:00.375217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:00.375184    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s" event={"ID":"0e5fb926-7759-4b04-9f9e-5c933d2c62d5","Type":"ContainerStarted","Data":"6d10528df90eb1d14eb4312388f5cab4d3e01ece8d1564183bb22c991d1eeab2"}
Oct 30 21:07:00.390830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:00.390782    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-z767s" podStartSLOduration=1.793735847 podStartE2EDuration="7.390769606s" totalImagesPullingTime="5.597033759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:00.390487057 +0000 UTC m=+277.346174285" watchObservedRunningTime="2026-10-30 21:07:00.390769606 +0000 UTC m=+277.346456853"
Oct 30 21:07:04.335616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:04.335582    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-nxr8f"
Oct 30 21:07:07.847616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:07.847583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:07.967423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:07.967393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:10.442926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:10.442891    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:10.566409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:10.566368    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.127128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.127093    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:12.132005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.131985    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.140794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.140482    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:12.152968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.152942    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.153114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.152975    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4gq2\" (UniqueName: \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.156709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.156692    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 21:07:12.169633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.169608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.223939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.223906    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:12.253920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.253894    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.254037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.253927    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4gq2\" (UniqueName: \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.254606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.254584    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.268997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.268976    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4gq2\" (UniqueName: \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\") pod \"limitador-limitador-74dfcfb894-lvz5n\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.345478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.345448    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.445013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.444928    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:12.468002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.467977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.580639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.580607    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:12.718271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.718146    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:12.720925 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:12.720901    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb4b557f_89a7_44df_8df1_d8b604f57e53.slice/crio-18fa1afeb252a21b5c5d187b9f8e9a5b5aa432843b69653e826dfdf84b6e672c WatchSource:0}: Error finding container 18fa1afeb252a21b5c5d187b9f8e9a5b5aa432843b69653e826dfdf84b6e672c: Status 404 returned error can't find the container with id 18fa1afeb252a21b5c5d187b9f8e9a5b5aa432843b69653e826dfdf84b6e672c
Oct 30 21:07:12.996968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:12.996904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:13.119991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:13.119958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:13.429605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:13.429578    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" event={"ID":"fb4b557f-89a7-44df-8df1-d8b604f57e53","Type":"ContainerStarted","Data":"18fa1afeb252a21b5c5d187b9f8e9a5b5aa432843b69653e826dfdf84b6e672c"}
Oct 30 21:07:13.644771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:13.644739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:13.786693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:13.786620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:15.445613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:15.445583    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" event={"ID":"fb4b557f-89a7-44df-8df1-d8b604f57e53","Type":"ContainerStarted","Data":"528e8d860485dc6b9c9c50af0dd027e8d18a5e9e2f22b6c79fc1e6d735333c7d"}
Oct 30 21:07:15.445983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:15.445719    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:15.462450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:15.462393    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" podStartSLOduration=1.721259603 podStartE2EDuration="3.462381495s" totalImagesPullingTime="1.741121892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:15.460939494 +0000 UTC m=+292.416626721" watchObservedRunningTime="2026-10-30 21:07:15.462381495 +0000 UTC m=+292.418068716"
Oct 30 21:07:16.986775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:16.986734    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.045148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:18.045115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.145562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:18.145532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.252661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:18.252640    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:18.347305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:18.347231    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:19.240077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:19.240045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:19.344458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:19.344413    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:22.505738 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:07:22.505703    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:07:26.026095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.026063    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:26.026481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.026306    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" podUID="fb4b557f-89a7-44df-8df1-d8b604f57e53" containerName="limitador" containerID="cri-o://528e8d860485dc6b9c9c50af0dd027e8d18a5e9e2f22b6c79fc1e6d735333c7d" gracePeriod=30
Oct 30 21:07:26.026938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.026919    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:26.491025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.490992    6738 generic.go:356] "Generic (PLEG): container finished" podID="fb4b557f-89a7-44df-8df1-d8b604f57e53" containerID="528e8d860485dc6b9c9c50af0dd027e8d18a5e9e2f22b6c79fc1e6d735333c7d" exitCode=0
Oct 30 21:07:26.491131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.491049    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" event={"ID":"fb4b557f-89a7-44df-8df1-d8b604f57e53","Type":"ContainerDied","Data":"528e8d860485dc6b9c9c50af0dd027e8d18a5e9e2f22b6c79fc1e6d735333c7d"}
Oct 30 21:07:26.855885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.855859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:26.953701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.953667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:26.959001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:26.958980    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:27.077725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.077699    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\" (UniqueName: \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\") pod \"fb4b557f-89a7-44df-8df1-d8b604f57e53\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") "
Oct 30 21:07:27.078043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.077786    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\" (UniqueName: \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\") pod \"fb4b557f-89a7-44df-8df1-d8b604f57e53\" (UID: \"fb4b557f-89a7-44df-8df1-d8b604f57e53\") "
Oct 30 21:07:27.078088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.078038    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file" pod "fb4b557f-89a7-44df-8df1-d8b604f57e53" (UID: "fb4b557f-89a7-44df-8df1-d8b604f57e53"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:07:27.079876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.079859    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2" pod "fb4b557f-89a7-44df-8df1-d8b604f57e53" (UID: "fb4b557f-89a7-44df-8df1-d8b604f57e53"). InnerVolumeSpecName "kube-api-access-h4gq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:07:27.178878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.178854    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4gq2\" (UniqueName: \"kubernetes.io/projected/fb4b557f-89a7-44df-8df1-d8b604f57e53-kube-api-access-h4gq2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:07:27.178878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.178875    6738 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fb4b557f-89a7-44df-8df1-d8b604f57e53-config-file\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:07:27.496198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.496145    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"
Oct 30 21:07:27.496198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.496159    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-lvz5n" event={"ID":"fb4b557f-89a7-44df-8df1-d8b604f57e53","Type":"ContainerDied","Data":"18fa1afeb252a21b5c5d187b9f8e9a5b5aa432843b69653e826dfdf84b6e672c"}
Oct 30 21:07:27.496333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.496209    6738 scope.go:122] "RemoveContainer" containerID="528e8d860485dc6b9c9c50af0dd027e8d18a5e9e2f22b6c79fc1e6d735333c7d"
Oct 30 21:07:27.511068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.511048    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:27.515129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:27.515111    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-lvz5n"]
Oct 30 21:07:29.161219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:29.161180    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb4b557f-89a7-44df-8df1-d8b604f57e53" path="/var/lib/kubelet/pods/fb4b557f-89a7-44df-8df1-d8b604f57e53/volumes"
Oct 30 21:07:31.961107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:31.961072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:32.073039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.073007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:32.604274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.604238    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"]
Oct 30 21:07:32.604999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.604971    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fb4b557f-89a7-44df-8df1-d8b604f57e53" containerName="limitador"
Oct 30 21:07:32.604999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.604998    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb4b557f-89a7-44df-8df1-d8b604f57e53" containerName="limitador"
Oct 30 21:07:32.605161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.605084    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb4b557f-89a7-44df-8df1-d8b604f57e53" containerName="limitador"
Oct 30 21:07:32.705000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.704974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:32.727898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.727869    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"]
Oct 30 21:07:32.728019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.727956    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.731178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.731157    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cq75f\""
Oct 30 21:07:32.731305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.731185    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 21:07:32.732302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.732289    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 21:07:32.732472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.732456    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 21:07:32.817507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.817483    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-data\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.817621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.817520    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msc8h\" (UniqueName: \"kubernetes.io/projected/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-kube-api-access-msc8h\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.918525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.918502    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-data\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.918628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.918537    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msc8h\" (UniqueName: \"kubernetes.io/projected/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-kube-api-access-msc8h\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.918814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.918796    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-data\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:32.926199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:32.926181    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msc8h\" (UniqueName: \"kubernetes.io/projected/6f9f2326-10fb-40e1-a2ad-772f38cc3c21-kube-api-access-msc8h\") pod \"postgres-7cdbf6dc44-xkl8h\" (UID: \"6f9f2326-10fb-40e1-a2ad-772f38cc3c21\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:33.066046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:33.066026    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:33.384195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:33.384123    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"]
Oct 30 21:07:33.386494 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:33.386466    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f9f2326_10fb_40e1_a2ad_772f38cc3c21.slice/crio-d4172cb17129bb97e836e42e7c2f5abc0beb20d150d667c437b5626ba83d2b25 WatchSource:0}: Error finding container d4172cb17129bb97e836e42e7c2f5abc0beb20d150d667c437b5626ba83d2b25: Status 404 returned error can't find the container with id d4172cb17129bb97e836e42e7c2f5abc0beb20d150d667c437b5626ba83d2b25
Oct 30 21:07:33.530707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:33.530678    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h" event={"ID":"6f9f2326-10fb-40e1-a2ad-772f38cc3c21","Type":"ContainerStarted","Data":"d4172cb17129bb97e836e42e7c2f5abc0beb20d150d667c437b5626ba83d2b25"}
Oct 30 21:07:39.560163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:39.560127    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h" event={"ID":"6f9f2326-10fb-40e1-a2ad-772f38cc3c21","Type":"ContainerStarted","Data":"7914c207b5f8f08b77339b53cbcfc1a7208b96482e9a2500bef9fc59cf885dcb"}
Oct 30 21:07:39.560644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:39.560249    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:39.575484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:39.575418    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h" podStartSLOduration=2.368396638 podStartE2EDuration="7.575404736s" totalImagesPullingTime="5.207008098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:39.574595773 +0000 UTC m=+316.530282998" watchObservedRunningTime="2026-10-30 21:07:39.575404736 +0000 UTC m=+316.531091962"
Oct 30 21:07:39.673215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:39.673194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:39.775154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:39.775125    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:45.594957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:45.594926    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xkl8h"
Oct 30 21:07:49.030678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.030641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:49.109370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.109339    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7tfz5"]
Oct 30 21:07:49.113161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.113144    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.120533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.120512    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 21:07:49.120695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.120670    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 21:07:49.122487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.122465    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7tfz5"]
Oct 30 21:07:49.130278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.130255    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:07:49.133115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.133095    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Oct 30 21:07:49.133201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.133181    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-hwttr\""
Oct 30 21:07:49.133680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.133660    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.146205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.146185    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:07:49.171501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.171480    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Oct 30 21:07:49.172020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.171993    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-m2cxx\""
Oct 30 21:07:49.181021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.181000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:49.184669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.184646    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-769875db56-s4vkj"]
Oct 30 21:07:49.188728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.188711    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.198054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.198032    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-metrics-tls\""
Oct 30 21:07:49.198054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.198032    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Oct 30 21:07:49.198223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.198131    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tlrl2\""
Oct 30 21:07:49.202472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.202425    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-s4vkj"]
Oct 30 21:07:49.256990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.256967    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-metrics-certs\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.257134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257111    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vq7s\" (UniqueName: \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.257393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257369    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-cert\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.257576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257557    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01001045-1325-4307-b938-8418266b3ec0-cert\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.257682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257584    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg689\" (UniqueName: \"kubernetes.io/projected/01001045-1325-4307-b938-8418266b3ec0-kube-api-access-bg689\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.257682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257649    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.257772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.257719    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f252n\" (UniqueName: \"kubernetes.io/projected/796c707d-027e-4303-aa99-6b46d94485d6-kube-api-access-f252n\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.314242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.314184    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:49.358966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.358936    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-metrics-certs\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.359072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.358994    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vq7s\" (UniqueName: \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.359143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.359070    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-cert\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.359209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.359191    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01001045-1325-4307-b938-8418266b3ec0-cert\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.359268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.359223    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg689\" (UniqueName: \"kubernetes.io/projected/01001045-1325-4307-b938-8418266b3ec0-kube-api-access-bg689\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.359324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.359266    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.359324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.359293    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f252n\" (UniqueName: \"kubernetes.io/projected/796c707d-027e-4303-aa99-6b46d94485d6-kube-api-access-f252n\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.361586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.361566    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-metrics-certs\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.361671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.361624    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796c707d-027e-4303-aa99-6b46d94485d6-cert\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.361801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.361782    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.361842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.361782    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01001045-1325-4307-b938-8418266b3ec0-cert\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.377906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.377879    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f252n\" (UniqueName: \"kubernetes.io/projected/796c707d-027e-4303-aa99-6b46d94485d6-kube-api-access-f252n\") pod \"odh-model-controller-769875db56-s4vkj\" (UID: \"796c707d-027e-4303-aa99-6b46d94485d6\") " pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.377906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.377901    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vq7s\" (UniqueName: \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\") pod \"llmisvc-controller-manager-869d5dcc5d-dxrzw\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.378126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.378104    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg689\" (UniqueName: \"kubernetes.io/projected/01001045-1325-4307-b938-8418266b3ec0-kube-api-access-bg689\") pod \"kserve-controller-manager-776cc66d47-7tfz5\" (UID: \"01001045-1325-4307-b938-8418266b3ec0\") " pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.428256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.428226    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:49.446647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.446556    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:49.447828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.447808    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:49.503881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.503848    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:49.733789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.733763    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-7tfz5"]
Oct 30 21:07:49.736133 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:49.736098    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01001045_1325_4307_b938_8418266b3ec0.slice/crio-86a7548af88913cad7d23951c697ac36152f009f3063c4e8b7c7b2052dedaaf6 WatchSource:0}: Error finding container 86a7548af88913cad7d23951c697ac36152f009f3063c4e8b7c7b2052dedaaf6: Status 404 returned error can't find the container with id 86a7548af88913cad7d23951c697ac36152f009f3063c4e8b7c7b2052dedaaf6
Oct 30 21:07:49.737448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.737419    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:07:49.757731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.757712    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:07:49.759193 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:49.759168    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod300aedb2_1033_48d9_8e2d_131da035df21.slice/crio-ca91455c892c60d3197878be7df9b8f5bd6034226dfa532c1ee1d044eb8bdd72 WatchSource:0}: Error finding container ca91455c892c60d3197878be7df9b8f5bd6034226dfa532c1ee1d044eb8bdd72: Status 404 returned error can't find the container with id ca91455c892c60d3197878be7df9b8f5bd6034226dfa532c1ee1d044eb8bdd72
Oct 30 21:07:49.810656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:49.810633    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-769875db56-s4vkj"]
Oct 30 21:07:49.812851 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:49.812827    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796c707d_027e_4303_aa99_6b46d94485d6.slice/crio-632b3517c78661d7bbbcf63c3bde637ad8644f7fdb3740896487736b7d9f61c8 WatchSource:0}: Error finding container 632b3517c78661d7bbbcf63c3bde637ad8644f7fdb3740896487736b7d9f61c8: Status 404 returned error can't find the container with id 632b3517c78661d7bbbcf63c3bde637ad8644f7fdb3740896487736b7d9f61c8
Oct 30 21:07:50.201954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.201925    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.312115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.312083    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.433692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.433654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:50.614261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.614221    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-s4vkj" event={"ID":"796c707d-027e-4303-aa99-6b46d94485d6","Type":"ContainerStarted","Data":"632b3517c78661d7bbbcf63c3bde637ad8644f7fdb3740896487736b7d9f61c8"}
Oct 30 21:07:50.617269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.617081    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" event={"ID":"300aedb2-1033-48d9-8e2d-131da035df21","Type":"ContainerStarted","Data":"ca91455c892c60d3197878be7df9b8f5bd6034226dfa532c1ee1d044eb8bdd72"}
Oct 30 21:07:50.619078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:50.619028    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5" event={"ID":"01001045-1325-4307-b938-8418266b3ec0","Type":"ContainerStarted","Data":"86a7548af88913cad7d23951c697ac36152f009f3063c4e8b7c7b2052dedaaf6"}
Oct 30 21:07:52.513179 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:07:52.513121    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:07:52.632786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.632756    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" event={"ID":"300aedb2-1033-48d9-8e2d-131da035df21","Type":"ContainerStarted","Data":"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"}
Oct 30 21:07:52.632936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.632910    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:07:52.650333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.650252    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" podStartSLOduration=0.934151332 podStartE2EDuration="3.650237309s" totalImagesPullingTime="2.716085977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:52.649622138 +0000 UTC m=+329.605309368" watchObservedRunningTime="2026-10-30 21:07:52.650237309 +0000 UTC m=+329.605924539"
Oct 30 21:07:52.789345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.789271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.845100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.845067    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:07:52.849721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.849699    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:52.853186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.853162    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Oct 30 21:07:52.853294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.853173    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Oct 30 21:07:52.853356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.853329    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Oct 30 21:07:52.853414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.853373    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-c2xvm\""
Oct 30 21:07:52.859466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.859418    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:07:52.899820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.899780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:52.994593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.994558    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:52.994775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.994676    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd2rc\" (UniqueName: \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:52.994775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:52.994737    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.008714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.008683    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:53.096718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.096654    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.096926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.096722    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd2rc\" (UniqueName: \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.096926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.096756    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.099953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.099931    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.100074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.099983    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.110125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.110083    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd2rc\" (UniqueName: \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\") pod \"maas-controller-7f9f487f56-ptzgq\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") " pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.167614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.167584    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:53.194142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.194115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:53.349501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.349418    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-c55679bf5-2hd6c"]
Oct 30 21:07:53.353026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.353008    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.364647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.364624    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c55679bf5-2hd6c"]
Oct 30 21:07:53.444416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.444372    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:07:53.446594 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:53.446568    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92bd87c8_7a07_42e2_847c_eeb7aa1e645a.slice/crio-cc7c556ccfc6d151b6f502cdeba5a0d93f47c887f78dca51afd2a5b2a54fcff4 WatchSource:0}: Error finding container cc7c556ccfc6d151b6f502cdeba5a0d93f47c887f78dca51afd2a5b2a54fcff4: Status 404 returned error can't find the container with id cc7c556ccfc6d151b6f502cdeba5a0d93f47c887f78dca51afd2a5b2a54fcff4
Oct 30 21:07:53.501199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.501174    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-metrics-certs\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.501490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.501235    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gfq\" (UniqueName: \"kubernetes.io/projected/ab742809-f65e-4524-aae4-bc1714674dda-kube-api-access-d2gfq\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.501490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.501395    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-webhook-cert\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.602941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.602864    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gfq\" (UniqueName: \"kubernetes.io/projected/ab742809-f65e-4524-aae4-bc1714674dda-kube-api-access-d2gfq\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.602941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.602931    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-webhook-cert\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.602941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.602999    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-metrics-certs\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.606243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.606214    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-metrics-certs\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.606820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.606789    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ab742809-f65e-4524-aae4-bc1714674dda-webhook-cert\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.611278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.611254    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gfq\" (UniqueName: \"kubernetes.io/projected/ab742809-f65e-4524-aae4-bc1714674dda-kube-api-access-d2gfq\") pod \"maas-controller-c55679bf5-2hd6c\" (UID: \"ab742809-f65e-4524-aae4-bc1714674dda\") " pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.638634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.638606    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-769875db56-s4vkj" event={"ID":"796c707d-027e-4303-aa99-6b46d94485d6","Type":"ContainerStarted","Data":"f8e3966baca0f03d9d3c5e160dc6fc6fcb6dfb7e0b5e09b978adf0ee70aeb88c"}
Oct 30 21:07:53.638780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.638763    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:07:53.640184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.640161    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5" event={"ID":"01001045-1325-4307-b938-8418266b3ec0","Type":"ContainerStarted","Data":"fc87125ad8412198dad618c25e5f70fc54f7bab1bd73f7c0b28a636f9aa4ee50"}
Oct 30 21:07:53.640297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.640268    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:07:53.641324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.641301    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" event={"ID":"92bd87c8-7a07-42e2-847c-eeb7aa1e645a","Type":"ContainerStarted","Data":"cc7c556ccfc6d151b6f502cdeba5a0d93f47c887f78dca51afd2a5b2a54fcff4"}
Oct 30 21:07:53.659096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.659055    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-model-controller-769875db56-s4vkj" podStartSLOduration=1.218854504 podStartE2EDuration="4.659041599s" totalImagesPullingTime="3.440187095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:53.657984954 +0000 UTC m=+330.613672183" watchObservedRunningTime="2026-10-30 21:07:53.659041599 +0000 UTC m=+330.614728826"
Oct 30 21:07:53.667985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.667964    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:53.674397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.674353    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5" podStartSLOduration=1.819462273 podStartE2EDuration="4.674342416s" totalImagesPullingTime="2.854880143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:53.67287035 +0000 UTC m=+330.628557579" watchObservedRunningTime="2026-10-30 21:07:53.674342416 +0000 UTC m=+330.630029665"
Oct 30 21:07:53.985757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:53.985725    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-c55679bf5-2hd6c"]
Oct 30 21:07:53.988965 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:07:53.988925    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab742809_f65e_4524_aae4_bc1714674dda.slice/crio-9fbd63574b68bca12fcd670458bcb0f997b2753b55f350877c77b00320a3d82a WatchSource:0}: Error finding container 9fbd63574b68bca12fcd670458bcb0f997b2753b55f350877c77b00320a3d82a: Status 404 returned error can't find the container with id 9fbd63574b68bca12fcd670458bcb0f997b2753b55f350877c77b00320a3d82a
Oct 30 21:07:54.651183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:54.651144    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c55679bf5-2hd6c" event={"ID":"ab742809-f65e-4524-aae4-bc1714674dda","Type":"ContainerStarted","Data":"9fbd63574b68bca12fcd670458bcb0f997b2753b55f350877c77b00320a3d82a"}
Oct 30 21:07:58.315662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.315630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:58.439583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.439542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:58.673191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.673156    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-c55679bf5-2hd6c" event={"ID":"ab742809-f65e-4524-aae4-bc1714674dda","Type":"ContainerStarted","Data":"c1dd1efb959dea664ab0199c635591924d3999d6c8ef65d622b57bd10de2295d"}
Oct 30 21:07:58.673359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.673337    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:07:58.674645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.674623    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" event={"ID":"92bd87c8-7a07-42e2-847c-eeb7aa1e645a","Type":"ContainerStarted","Data":"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"}
Oct 30 21:07:58.674763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.674752    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:07:58.693130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.693041    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-c55679bf5-2hd6c" podStartSLOduration=1.581547097 podStartE2EDuration="5.693029416s" totalImagesPullingTime="4.111482319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:58.691156648 +0000 UTC m=+335.646843877" watchObservedRunningTime="2026-10-30 21:07:58.693029416 +0000 UTC m=+335.648716705"
Oct 30 21:07:58.708253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:58.708205    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" podStartSLOduration=2.068376971 podStartE2EDuration="6.708191463s" totalImagesPullingTime="4.639814492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:07:58.706763933 +0000 UTC m=+335.662451162" watchObservedRunningTime="2026-10-30 21:07:58.708191463 +0000 UTC m=+335.663878691"
Oct 30 21:07:59.776258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.776219    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"]
Oct 30 21:07:59.780223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.780195    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:07:59.786216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.786188    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 21:07:59.790044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.790020    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-vkqt2\""
Oct 30 21:07:59.790560 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.790378    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 21:07:59.790560 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.790514    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 21:07:59.792726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.792697    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"]
Oct 30 21:07:59.861198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.861165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:59.960487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.960455    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:07:59.961594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.961574    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:07:59.961660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.961608    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:07:59.961660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.961637    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e53aa576-fca2-4a2c-944d-cf252d4f239c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:07:59.961829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:07:59.961809    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxcxr\" (UniqueName: \"kubernetes.io/projected/e53aa576-fca2-4a2c-944d-cf252d4f239c-kube-api-access-mxcxr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.062606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.062540    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e53aa576-fca2-4a2c-944d-cf252d4f239c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.062606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.062598    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxcxr\" (UniqueName: \"kubernetes.io/projected/e53aa576-fca2-4a2c-944d-cf252d4f239c-kube-api-access-mxcxr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.062763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.062629    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.062763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.062649    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.063404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.063375    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.063641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.063624    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/e53aa576-fca2-4a2c-944d-cf252d4f239c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.065151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.065130    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e53aa576-fca2-4a2c-944d-cf252d4f239c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.070009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.069983    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxcxr\" (UniqueName: \"kubernetes.io/projected/e53aa576-fca2-4a2c-944d-cf252d4f239c-kube-api-access-mxcxr\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-s2cll\" (UID: \"e53aa576-fca2-4a2c-944d-cf252d4f239c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.097019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.097001    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:00.371083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.371058    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"]
Oct 30 21:08:00.373316 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:08:00.373288    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode53aa576_fca2_4a2c_944d_cf252d4f239c.slice/crio-196820bcacc961ad4a61bb2b0622d1d85c454e26c10cc50f2ba0ac9368b6e88e WatchSource:0}: Error finding container 196820bcacc961ad4a61bb2b0622d1d85c454e26c10cc50f2ba0ac9368b6e88e: Status 404 returned error can't find the container with id 196820bcacc961ad4a61bb2b0622d1d85c454e26c10cc50f2ba0ac9368b6e88e
Oct 30 21:08:00.685471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:00.685362    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll" event={"ID":"e53aa576-fca2-4a2c-944d-cf252d4f239c","Type":"ContainerStarted","Data":"196820bcacc961ad4a61bb2b0622d1d85c454e26c10cc50f2ba0ac9368b6e88e"}
Oct 30 21:08:04.654592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:04.654561    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-769875db56-s4vkj"
Oct 30 21:08:09.681581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:09.681548    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-c55679bf5-2hd6c"
Oct 30 21:08:09.681988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:09.681643    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:08:09.739162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:09.739124    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:08:09.745196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:09.744844    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" podUID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" containerName="manager" containerID="cri-o://930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899" gracePeriod=30
Oct 30 21:08:10.027241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.027216    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:08:10.059519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.059487    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\" (UniqueName: \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\") pod \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") "
Oct 30 21:08:10.059725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.059574    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\") pod \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") "
Oct 30 21:08:10.059725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.059654    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\") pod \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\" (UID: \"92bd87c8-7a07-42e2-847c-eeb7aa1e645a\") "
Oct 30 21:08:10.062939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.062894    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs" pod "92bd87c8-7a07-42e2-847c-eeb7aa1e645a" (UID: "92bd87c8-7a07-42e2-847c-eeb7aa1e645a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:08:10.063213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.063190    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc" pod "92bd87c8-7a07-42e2-847c-eeb7aa1e645a" (UID: "92bd87c8-7a07-42e2-847c-eeb7aa1e645a"). InnerVolumeSpecName "kube-api-access-wd2rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:08:10.063415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.063384    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert" pod "92bd87c8-7a07-42e2-847c-eeb7aa1e645a" (UID: "92bd87c8-7a07-42e2-847c-eeb7aa1e645a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:08:10.160725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.160690    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd2rc\" (UniqueName: \"kubernetes.io/projected/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-kube-api-access-wd2rc\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:10.160725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.160725    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:10.160947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.160736    6738 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/92bd87c8-7a07-42e2-847c-eeb7aa1e645a-webhook-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:08:10.751633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.751593    6738 generic.go:356] "Generic (PLEG): container finished" podID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" containerID="930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899" exitCode=0
Oct 30 21:08:10.752123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.751671    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f9f487f56-ptzgq"
Oct 30 21:08:10.752123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.751688    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" event={"ID":"92bd87c8-7a07-42e2-847c-eeb7aa1e645a","Type":"ContainerDied","Data":"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"}
Oct 30 21:08:10.752123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.751757    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f9f487f56-ptzgq" event={"ID":"92bd87c8-7a07-42e2-847c-eeb7aa1e645a","Type":"ContainerDied","Data":"cc7c556ccfc6d151b6f502cdeba5a0d93f47c887f78dca51afd2a5b2a54fcff4"}
Oct 30 21:08:10.752123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.751783    6738 scope.go:122] "RemoveContainer" containerID="930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"
Oct 30 21:08:10.763031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.763001    6738 scope.go:122] "RemoveContainer" containerID="930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"
Oct 30 21:08:10.763339 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:08:10.763321    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899\": container with ID starting with 930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899 not found: ID does not exist" containerID="930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"
Oct 30 21:08:10.763403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.763347    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899"} err="failed to get container status \"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899\": rpc error: code = NotFound desc = could not find container \"930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899\": container with ID starting with 930a18e10ff0dfb849daa72540b65a879a1db730fa89ce2f9d97893cae89e899 not found: ID does not exist"
Oct 30 21:08:10.774564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.774537    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:08:10.780271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:10.780239    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f9f487f56-ptzgq"]
Oct 30 21:08:11.162230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:11.162196    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" path="/var/lib/kubelet/pods/92bd87c8-7a07-42e2-847c-eeb7aa1e645a/volumes"
Oct 30 21:08:12.769939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:12.769902    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll" event={"ID":"e53aa576-fca2-4a2c-944d-cf252d4f239c","Type":"ContainerStarted","Data":"14cdd950ff88b896e9bd8c8b2611cd944eb9647f02f710fe4698243589810412"}
Oct 30 21:08:12.770295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:12.770057    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:12.788004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:12.787963    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll" podStartSLOduration=2.383360851 podStartE2EDuration="13.787949994s" totalImagesPullingTime="11.404589143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:12.786741772 +0000 UTC m=+349.742429000" watchObservedRunningTime="2026-10-30 21:08:12.787949994 +0000 UTC m=+349.743637222"
Oct 30 21:08:17.062123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:17.062095    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.735983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:21.735941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:21.854947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:21.854915    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:22.507138 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:08:22.507107    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:08:23.643965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:23.643938    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:08:23.780220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:23.780190    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-s2cll"
Oct 30 21:08:24.656428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:24.656398    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-7tfz5"
Oct 30 21:08:30.112916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:30.112884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:30.245121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:30.245085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:31.440654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:31.440618    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:31.568782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:31.568726    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:31.674191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:31.674161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:43.913461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:43.913364    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:44.116367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.116339    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:44.413179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.413147    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"]
Oct 30 21:08:44.414448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.414401    6738 cpu_manager.go:414] "containerMap: removing container" podUID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" containerName="manager"
Oct 30 21:08:44.414626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.414609    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" containerName="manager"
Oct 30 21:08:44.415017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.415000    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="92bd87c8-7a07-42e2-847c-eeb7aa1e645a" containerName="manager"
Oct 30 21:08:44.443997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.443967    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"]
Oct 30 21:08:44.444123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.444075    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.459399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.459379    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rx79k\""
Oct 30 21:08:44.459399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.459384    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 21:08:44.459614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.459599    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 21:08:44.558824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.558802    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-metrics-certs\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.558955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.558838    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-openshift-service-ca\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.558955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.558883    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmdkg\" (UniqueName: \"kubernetes.io/projected/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-kube-api-access-hmdkg\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.558955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.558922    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-maas-api-tls\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.659862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.659838    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-openshift-service-ca\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.660009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.659866    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmdkg\" (UniqueName: \"kubernetes.io/projected/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-kube-api-access-hmdkg\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.660009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.659898    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-maas-api-tls\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.660009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.659960    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-metrics-certs\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.660697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.660678    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-openshift-service-ca\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.663041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.663023    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-metrics-certs\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.663223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.663178    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-maas-api-tls\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.667942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.667920    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmdkg\" (UniqueName: \"kubernetes.io/projected/c4ee81fb-a3c1-4ada-b749-7ab223d1ef84-kube-api-access-hmdkg\") pod \"maas-api-6b56594c78-wsdp7\" (UID: \"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84\") " pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:44.757525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:44.757504    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:45.027700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:45.027609    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"]
Oct 30 21:08:45.030576 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:08:45.030540    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ee81fb_a3c1_4ada_b749_7ab223d1ef84.slice/crio-555ca48193ca3288e03ec44524a163c896a590945152ed49ce529ac33ad6f368 WatchSource:0}: Error finding container 555ca48193ca3288e03ec44524a163c896a590945152ed49ce529ac33ad6f368: Status 404 returned error can't find the container with id 555ca48193ca3288e03ec44524a163c896a590945152ed49ce529ac33ad6f368
Oct 30 21:08:45.922579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:45.922544    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7" event={"ID":"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84","Type":"ContainerStarted","Data":"555ca48193ca3288e03ec44524a163c896a590945152ed49ce529ac33ad6f368"}
Oct 30 21:08:46.929165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:46.929128    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7" event={"ID":"c4ee81fb-a3c1-4ada-b749-7ab223d1ef84","Type":"ContainerStarted","Data":"9e7a1bdaa0ef601306e7c20c1026c1ab552585e6917b4a902d84723b2cd4b790"}
Oct 30 21:08:46.929716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:46.929211    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:08:46.947515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:46.947466    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7" podStartSLOduration=1.617965258 podStartE2EDuration="2.947453169s" totalImagesPullingTime="1.329487911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:46.945984116 +0000 UTC m=+383.901671346" watchObservedRunningTime="2026-10-30 21:08:46.947453169 +0000 UTC m=+383.903140400"
Oct 30 21:08:47.547127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.547093    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"]
Oct 30 21:08:47.550803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.550782    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.559535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.559509    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"]
Oct 30 21:08:47.585892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.585868    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6v88\" (UniqueName: \"kubernetes.io/projected/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-kube-api-access-g6v88\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.586016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.585901    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-cert\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.687208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.687181    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6v88\" (UniqueName: \"kubernetes.io/projected/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-kube-api-access-g6v88\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.687332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.687217    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-cert\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.689783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.689760    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-cert\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.695032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.695010    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6v88\" (UniqueName: \"kubernetes.io/projected/6b4f1153-0be9-417b-bd15-8cdb5bdaa169-kube-api-access-g6v88\") pod \"llmisvc-controller-manager-6466c554d8-hzj4m\" (UID: \"6b4f1153-0be9-417b-bd15-8cdb5bdaa169\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:47.865556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:47.865530    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:48.137364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:48.137336    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"]
Oct 30 21:08:48.941997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:48.941960    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m" event={"ID":"6b4f1153-0be9-417b-bd15-8cdb5bdaa169","Type":"ContainerStarted","Data":"2693151abf1ea37dfced523d3e6ea47587b9dfe80dddfb80a259ca66a5b1cd42"}
Oct 30 21:08:48.942182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:48.942046    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m" event={"ID":"6b4f1153-0be9-417b-bd15-8cdb5bdaa169","Type":"ContainerStarted","Data":"75baf7b88c040e4d41ce26643faa8690432673b89ec303b095f034e68764a61c"}
Oct 30 21:08:48.942182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:48.942081    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:08:48.957085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:48.957031    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m" podStartSLOduration=1.5526985 podStartE2EDuration="1.957016289s" totalImagesPullingTime="404.317789ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:08:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:08:48.95655939 +0000 UTC m=+385.912246623" watchObservedRunningTime="2026-10-30 21:08:48.957016289 +0000 UTC m=+385.912703519"
Oct 30 21:08:51.581343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:51.581310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:51.681846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:51.681789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:08:52.507007 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:08:52.506961    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:08:52.941716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:08:52.941691    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b56594c78-wsdp7"
Oct 30 21:09:17.101196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:17.101161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:19.948892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:19.948865    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-hzj4m"
Oct 30 21:09:19.989975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:19.989940    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:09:19.990246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:19.990200    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" podUID="300aedb2-1033-48d9-8e2d-131da035df21" containerName="manager" containerID="cri-o://990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b" gracePeriod=30
Oct 30 21:09:20.239495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.239470    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:09:20.359892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.359863    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\" (UniqueName: \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\") pod \"300aedb2-1033-48d9-8e2d-131da035df21\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") "
Oct 30 21:09:20.360051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.359942    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\" (UniqueName: \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\") pod \"300aedb2-1033-48d9-8e2d-131da035df21\" (UID: \"300aedb2-1033-48d9-8e2d-131da035df21\") "
Oct 30 21:09:20.362157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.362125    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s" pod "300aedb2-1033-48d9-8e2d-131da035df21" (UID: "300aedb2-1033-48d9-8e2d-131da035df21"). InnerVolumeSpecName "kube-api-access-6vq7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:09:20.362265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.362160    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert" pod "300aedb2-1033-48d9-8e2d-131da035df21" (UID: "300aedb2-1033-48d9-8e2d-131da035df21"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:09:20.461418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.461392    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vq7s\" (UniqueName: \"kubernetes.io/projected/300aedb2-1033-48d9-8e2d-131da035df21-kube-api-access-6vq7s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:20.461418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:20.461414    6738 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/300aedb2-1033-48d9-8e2d-131da035df21-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:09:21.090756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.090721    6738 generic.go:356] "Generic (PLEG): container finished" podID="300aedb2-1033-48d9-8e2d-131da035df21" containerID="990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b" exitCode=0
Oct 30 21:09:21.091134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.090775    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"
Oct 30 21:09:21.091134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.090802    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" event={"ID":"300aedb2-1033-48d9-8e2d-131da035df21","Type":"ContainerDied","Data":"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"}
Oct 30 21:09:21.091134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.090857    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw" event={"ID":"300aedb2-1033-48d9-8e2d-131da035df21","Type":"ContainerDied","Data":"ca91455c892c60d3197878be7df9b8f5bd6034226dfa532c1ee1d044eb8bdd72"}
Oct 30 21:09:21.091134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.090878    6738 scope.go:122] "RemoveContainer" containerID="990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"
Oct 30 21:09:21.100311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.100292    6738 scope.go:122] "RemoveContainer" containerID="990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"
Oct 30 21:09:21.100611 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:09:21.100592    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b\": container with ID starting with 990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b not found: ID does not exist" containerID="990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"
Oct 30 21:09:21.100679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.100620    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b"} err="failed to get container status \"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b\": rpc error: code = NotFound desc = could not find container \"990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b\": container with ID starting with 990b870cefd5851aed027443842cde2b7627dd74a48f4b07ba38751d711f420b not found: ID does not exist"
Oct 30 21:09:21.114466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.114428    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:09:21.119084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.119066    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-dxrzw"]
Oct 30 21:09:21.162235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.162215    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="300aedb2-1033-48d9-8e2d-131da035df21" path="/var/lib/kubelet/pods/300aedb2-1033-48d9-8e2d-131da035df21/volumes"
Oct 30 21:09:21.372492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.372462    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:21.496447 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:21.496416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:22.506034 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:09:22.505999    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:09:24.355959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:24.355922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:24.478548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:24.478522    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:49.612225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:49.612192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:50.140671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:50.140643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:51.468624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:51.468590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:51.591127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:51.591097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:52.506787 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:09:52.506753    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:09:54.753654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:54.753620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:58.790779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:58.790734    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:09:59.587661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:09:59.587598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:00.420299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:00.420264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:01.142919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:01.142880    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:02.779576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:02.779525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:06.162265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:06.162184    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:07.705953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:07.705920    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:17.144657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:17.144626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:22.505887 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:10:22.505854    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:10:26.421719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:26.421684    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:26.564797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:26.564757    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:28.129563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.129517    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:28.270288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.270258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:10:28.735665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.735638    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:10:28.736072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.736058    6738 cpu_manager.go:414] "containerMap: removing container" podUID="300aedb2-1033-48d9-8e2d-131da035df21" containerName="manager"
Oct 30 21:10:28.736158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.736074    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="300aedb2-1033-48d9-8e2d-131da035df21" containerName="manager"
Oct 30 21:10:28.736158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.736150    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="300aedb2-1033-48d9-8e2d-131da035df21" containerName="manager"
Oct 30 21:10:28.747679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.746520    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.753897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.753871    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:10:28.754865 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.754819    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 21:10:28.787040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.787009    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-l56fx\""
Oct 30 21:10:28.845949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.845921    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.845956    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.845973    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.845989    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.846094    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.846175    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.846232    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.846344    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5b8w2\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.846402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.846377    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947580    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947620    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947682    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5b8w2\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947723    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947777    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947802    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947823    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947845    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.947991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.947883    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.948198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.948143    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.948255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.948206    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.948310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.948287    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.948367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.948342    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.948726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.948703    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.950416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.950389    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.951268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.951237    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.956051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.956026    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5b8w2\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:28.956461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:28.956420    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-zqthr\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:29.065086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:29.065028    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:29.340941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:29.340869    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:10:29.344901 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:10:29.344871    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039b6249_f141_4c9f_9e6d_3c1e2f297d76.slice/crio-2ca00add4c2997c6fe160dac87e5ab5f1ba121cf96a0f163c9fc3d3421e5f269 WatchSource:0}: Error finding container 2ca00add4c2997c6fe160dac87e5ab5f1ba121cf96a0f163c9fc3d3421e5f269: Status 404 returned error can't find the container with id 2ca00add4c2997c6fe160dac87e5ab5f1ba121cf96a0f163c9fc3d3421e5f269
Oct 30 21:10:29.419059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:29.419028    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" event={"ID":"039b6249-f141-4c9f-9e6d-3c1e2f297d76","Type":"ContainerStarted","Data":"2ca00add4c2997c6fe160dac87e5ab5f1ba121cf96a0f163c9fc3d3421e5f269"}
Oct 30 21:10:32.585533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:32.585500    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:32.585784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:32.585577    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:32.585784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:32.585608    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:33.442949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:33.442915    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" event={"ID":"039b6249-f141-4c9f-9e6d-3c1e2f297d76","Type":"ContainerStarted","Data":"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"}
Oct 30 21:10:34.066030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:34.065997    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:34.066962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:34.066936    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:34.067017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:34.066988    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:35.066203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:35.066172    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:35.066581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:35.066230    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:36.065978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:36.065939    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:36.066132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:36.066012    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:37.066193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:37.066158    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:37.066614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:37.066208    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:38.066295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:38.066262    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:38.066696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:38.066317    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:39.065584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:39.065548    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 21:10:39.065755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:39.065601    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 21:10:40.694720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:40.694692    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:40.695276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:40.695258    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:40.696366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:40.696347    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:10:40.715547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:40.715504    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podStartSLOduration=9.47823831 podStartE2EDuration="12.715492306s" totalImagesPullingTime="3.237253996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:33.464475148 +0000 UTC m=+490.420162379" watchObservedRunningTime="2026-10-30 21:10:40.715492306 +0000 UTC m=+497.671179534"
Oct 30 21:10:43.772572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.772543    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"]
Oct 30 21:10:43.775677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.775655    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.790626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.790603    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"]
Oct 30 21:10:43.805058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805034    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zln4l\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-kube-api-access-zln4l\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805068    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805161    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04cf628-eece-472b-91c0-8b25c2407e57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805238    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805299    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805333    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805356    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04cf628-eece-472b-91c0-8b25c2407e57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805375    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.805498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.805484    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.905976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.905953    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.905995    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906023    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906053    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04cf628-eece-472b-91c0-8b25c2407e57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906082    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906113    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906249    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zln4l\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-kube-api-access-zln4l\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906309    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906392    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04cf628-eece-472b-91c0-8b25c2407e57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906466    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906389    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906582    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.906738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.906647    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.907045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.907025    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a04cf628-eece-472b-91c0-8b25c2407e57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.908478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.908452    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a04cf628-eece-472b-91c0-8b25c2407e57-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.908784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.908761    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a04cf628-eece-472b-91c0-8b25c2407e57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.913941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.913919    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:43.914033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:43.913990    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zln4l\" (UniqueName: \"kubernetes.io/projected/a04cf628-eece-472b-91c0-8b25c2407e57-kube-api-access-zln4l\") pod \"maas-default-gateway-openshift-default-75bf867b6c-xcwwf\" (UID: \"a04cf628-eece-472b-91c0-8b25c2407e57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:44.090983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.090911    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:44.374285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.374260    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"]
Oct 30 21:10:44.377171 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:10:44.377146    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda04cf628_eece_472b_91c0_8b25c2407e57.slice/crio-8289f04d29730f5b47d0adf42713370291060ee995b52c6150aa103ba0208165 WatchSource:0}: Error finding container 8289f04d29730f5b47d0adf42713370291060ee995b52c6150aa103ba0208165: Status 404 returned error can't find the container with id 8289f04d29730f5b47d0adf42713370291060ee995b52c6150aa103ba0208165
Oct 30 21:10:44.380916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.380879    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:44.381015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.380952    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:44.381015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.380979    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:44.496364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.496326    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" event={"ID":"a04cf628-eece-472b-91c0-8b25c2407e57","Type":"ContainerStarted","Data":"9a8af6acbfc88d8b4e240fcba5efe7c48d1a624ee22268d4fdb3721977fe8e1f"}
Oct 30 21:10:44.496543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.496391    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" event={"ID":"a04cf628-eece-472b-91c0-8b25c2407e57","Type":"ContainerStarted","Data":"8289f04d29730f5b47d0adf42713370291060ee995b52c6150aa103ba0208165"}
Oct 30 21:10:44.517106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:44.517047    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podStartSLOduration=1.5170269269999999 podStartE2EDuration="1.517026927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:44.514616247 +0000 UTC m=+501.470303477" watchObservedRunningTime="2026-10-30 21:10:44.517026927 +0000 UTC m=+501.472714155"
Oct 30 21:10:45.102042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:45.098007    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:45.107469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:45.105559    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:45.107469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:45.105623    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:46.096461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:46.091772    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:46.096461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:46.091829    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:47.094467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:47.093833    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:47.094467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:47.093908    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:48.107830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:48.107793    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:48.109244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:48.108878    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:49.095470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:49.092792    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:49.095470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:49.092863    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:50.099885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:50.099842    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:50.100541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:50.100506    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:51.092194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:51.092156    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 21:10:51.092393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:51.092212    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 21:10:52.511809 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:10:52.511748    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:10:52.547921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:52.547886    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:52.548332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:52.548313    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:52.549235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:52.549201    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf"
Oct 30 21:10:58.778318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.778285    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"]
Oct 30 21:10:58.782148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.782121    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.796822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.796796    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"]
Oct 30 21:10:58.846130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846100    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846133    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b791e110-a436-4eba-9674-7c0fded22180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846151    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b791e110-a436-4eba-9674-7c0fded22180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846269    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846315    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846349    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t75m8\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-kube-api-access-t75m8\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846468    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846502    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.846599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.846526    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947320    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947360    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947377    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b791e110-a436-4eba-9674-7c0fded22180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947393    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b791e110-a436-4eba-9674-7c0fded22180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947460    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947508    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947536    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t75m8\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-kube-api-access-t75m8\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947581    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947626    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947841    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.947972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947936    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.948072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.947943    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.948072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.948034    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.948194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.948168    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b791e110-a436-4eba-9674-7c0fded22180-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.949893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.949873    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b791e110-a436-4eba-9674-7c0fded22180-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.949982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.949939    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b791e110-a436-4eba-9674-7c0fded22180-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.955510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.955489    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:58.955691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:58.955674    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t75m8\" (UniqueName: \"kubernetes.io/projected/b791e110-a436-4eba-9674-7c0fded22180-kube-api-access-t75m8\") pod \"maas-default-gateway-openshift-default-75bf867b6c-vgrr4\" (UID: \"b791e110-a436-4eba-9674-7c0fded22180\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:59.095962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.095909    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:10:59.372314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.372287    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"]
Oct 30 21:10:59.374637 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:10:59.374609    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb791e110_a436_4eba_9674_7c0fded22180.slice/crio-7fb40c7b81f8619eec9607b1e511cc73ff2ef777824c1c114eff4e594bf54920 WatchSource:0}: Error finding container 7fb40c7b81f8619eec9607b1e511cc73ff2ef777824c1c114eff4e594bf54920: Status 404 returned error can't find the container with id 7fb40c7b81f8619eec9607b1e511cc73ff2ef777824c1c114eff4e594bf54920
Oct 30 21:10:59.376781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.376750    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:59.376848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.376816    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:59.376889 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.376846    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:10:59.587817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.587786    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" event={"ID":"b791e110-a436-4eba-9674-7c0fded22180","Type":"ContainerStarted","Data":"9d24d53feef0d28e21f70ab75e0060c67b96a3c3a6e48aed98aa06a4a7521c6e"}
Oct 30 21:10:59.587982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:10:59.587843    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" event={"ID":"b791e110-a436-4eba-9674-7c0fded22180","Type":"ContainerStarted","Data":"7fb40c7b81f8619eec9607b1e511cc73ff2ef777824c1c114eff4e594bf54920"}
Oct 30 21:11:00.103461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:00.097810    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:11:00.112605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:00.112570    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:00.112904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:00.112865    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:01.112593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:01.112554    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:01.113013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:01.112607    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:02.096726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:02.096689    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:02.096998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:02.096971    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:03.096780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:03.096746    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:03.097367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:03.097335    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:04.100850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.100810    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:04.101460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.101405    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:04.535328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.535262    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.668303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.668271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.808480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.808346    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:04.929598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:04.929565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.096842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.096756    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:05.096842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.096817    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:05.155059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.155007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.286324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.286286    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.406495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.406454    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.536292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.536257    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.657904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.657807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:05.772469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:05.772406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:06.097297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:06.097205    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Oct 30 21:11:06.097297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:06.097285    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Oct 30 21:11:06.609917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:06.609883    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.058333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.058294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.177185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.177139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:07.755248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.755208    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:11:07.755683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.755663    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:11:07.757062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.757040    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4"
Oct 30 21:11:07.774872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:07.774824    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podStartSLOduration=9.774810132 podStartE2EDuration="9.774810132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:10:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:10:59.610919152 +0000 UTC m=+516.566606381" watchObservedRunningTime="2026-10-30 21:11:07.774810132 +0000 UTC m=+524.730497360"
Oct 30 21:11:08.007248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.007167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.170861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.170820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.281559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.281478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.409445 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.409397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.526651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.526617    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:08.928408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:08.928374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.057361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:09.057327    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.338347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:09.338260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.468282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:09.468252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.632384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:09.632318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:09.750590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:09.750531    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.140860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:10.140832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.251274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:10.251239    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.504342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:10.504262    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:10.645480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:10.645422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.289099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:11.289063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.414761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:11.414729    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:11.555987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:11.555905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.062323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.062284    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:12.066107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.066089    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.069297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.069278    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e2cef583-serving-cert\""
Oct 30 21:11:12.077787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.077762    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:12.133236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.133205    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.204623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.204587    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.204796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.204635    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.204796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.204757    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.204892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.204876    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rktxg\" (UniqueName: \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.243926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.243893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.306092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.306057    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.306538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.306120    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.306538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.306176    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.306538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.306253    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rktxg\" (UniqueName: \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.307177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.307154    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.308780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.308759    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.308874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.308814    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.321847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.321784    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rktxg\" (UniqueName: \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\") pod \"maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.371366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.371345    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:12.382316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.382304    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:12.657948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.657910    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:12.662372 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:11:12.662339    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fdc740e_7cf5_4b19_b7af_4cd2beed625c.slice/crio-a77a1eeabbf57984083ad58f0903bb7a61a9745e03c883ef3b9bd7ebe54cc8af WatchSource:0}: Error finding container a77a1eeabbf57984083ad58f0903bb7a61a9745e03c883ef3b9bd7ebe54cc8af: Status 404 returned error can't find the container with id a77a1eeabbf57984083ad58f0903bb7a61a9745e03c883ef3b9bd7ebe54cc8af
Oct 30 21:11:12.687905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:12.687872    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" event={"ID":"5fdc740e-7cf5-4b19-b7af-4cd2beed625c","Type":"ContainerStarted","Data":"a77a1eeabbf57984083ad58f0903bb7a61a9745e03c883ef3b9bd7ebe54cc8af"}
Oct 30 21:11:13.042588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.042509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.168792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.168763    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.307825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.307727    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.436961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.436916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:13.696325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.696284    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" event={"ID":"5fdc740e-7cf5-4b19-b7af-4cd2beed625c","Type":"ContainerStarted","Data":"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"}
Oct 30 21:11:13.696579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.696527    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:13.747642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.747592    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" podStartSLOduration=1.290010631 podStartE2EDuration="1.747576498s" totalImagesPullingTime="457.565867ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:13.746232434 +0000 UTC m=+530.701919663" watchObservedRunningTime="2026-10-30 21:11:13.747576498 +0000 UTC m=+530.703263752"
Oct 30 21:11:13.955176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:13.955097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.083178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.083140    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.207283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.207209    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.668553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.668522    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:14.672634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.672608    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.682248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.682223    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-111fd691-serving-cert\""
Oct 30 21:11:14.684106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.683480    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:14.732265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.732234    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccbbc\" (UniqueName: \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.732404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.732276    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.732552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.732519    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.732644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.732575    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.737159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.737138    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.833479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.833421    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.833661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.833539    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccbbc\" (UniqueName: \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.833661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.833607    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.833784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.833737    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.834509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.834481    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.836127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.836103    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.836212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.836164    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.845274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.845248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccbbc\" (UniqueName: \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\") pod \"maas-api-e2e-ait-111fd691-867cdcc74b-qwprs\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:14.856603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.856574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.963753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.963695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:14.994355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:14.994330    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:15.274856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:15.274815    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:15.277040 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:11:15.277008    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c24117_dbe8_4d5d_953c_7f480f5b68f4.slice/crio-938d02b9903fd88749e72ca28a6d129ee427985f9c9296eaf31868ceec4d5ee9 WatchSource:0}: Error finding container 938d02b9903fd88749e72ca28a6d129ee427985f9c9296eaf31868ceec4d5ee9: Status 404 returned error can't find the container with id 938d02b9903fd88749e72ca28a6d129ee427985f9c9296eaf31868ceec4d5ee9
Oct 30 21:11:15.711096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:15.711044    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" event={"ID":"04c24117-dbe8-4d5d-953c-7f480f5b68f4","Type":"ContainerStarted","Data":"938d02b9903fd88749e72ca28a6d129ee427985f9c9296eaf31868ceec4d5ee9"}
Oct 30 21:11:15.716247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:15.716220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.264903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.264871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.396806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.396767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:16.719205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.719165    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" event={"ID":"04c24117-dbe8-4d5d-953c-7f480f5b68f4","Type":"ContainerStarted","Data":"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"}
Oct 30 21:11:16.719610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.719280    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:16.737974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.737931    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" podStartSLOduration=2.305291097 podStartE2EDuration="2.737917159s" totalImagesPullingTime="432.626062ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:16.736384156 +0000 UTC m=+533.692071384" watchObservedRunningTime="2026-10-30 21:11:16.737917159 +0000 UTC m=+533.693604386"
Oct 30 21:11:16.977644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:16.977569    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.223048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:17.223014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.410207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:17.410162    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.543482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:17.543451    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:17.940479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:17.940414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:18.935775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:18.935738    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:19.709142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:19.709112    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:19.814561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:19.814530    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:20.544677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:20.544638    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:20.651845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:20.651814    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:20.951939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:20.951909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:21.229626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:21.229546    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:21.997093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:21.997058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:22.508854 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:22.508816    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:11:22.730328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:22.730296    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:22.996072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:22.996036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:23.916562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:23.916530    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:24.877180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:24.877149    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:25.586212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:25.586180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:26.347826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:26.347790    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:27.036549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:27.036513    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:27.742591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:27.742560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:28.552220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:28.552185    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:29.723399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:29.723370    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:30.544458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:30.544408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:31.312926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:31.312897    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:32.110857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:32.110820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:33.028796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:33.028761    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:33.449131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:33.449081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:33.571738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:33.571695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:34.029877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:34.029845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:34.727943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:34.727857    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:35.535849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:35.535818    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.077389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:36.077356    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.518466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:36.518403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.660824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:36.660723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:36.774172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:36.774061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:37.581481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:37.581449    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:37.688125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:37.688085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:38.056017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:38.055984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:38.172477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:38.172416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:38.600622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:38.600576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.349057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:39.349021    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.463255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:39.463218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.842213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:39.842178    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:39.959375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:39.959345    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:40.288326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:40.288293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:40.789229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:40.789198    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:40.790055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:40.789484    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" podUID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" containerName="maas-api" containerID="cri-o://5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248" gracePeriod=30
Oct 30 21:11:40.851148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:40.851114    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:40.999460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:40.993542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.107891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.107862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.129411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.129385    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:41.207636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.207142    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\") pod \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") "
Oct 30 21:11:41.207636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.207227    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\") pod \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") "
Oct 30 21:11:41.207636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.207306    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\") pod \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") "
Oct 30 21:11:41.207636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.207362    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\" (UniqueName: \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\") pod \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\" (UID: \"5fdc740e-7cf5-4b19-b7af-4cd2beed625c\") "
Oct 30 21:11:41.215504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.215385    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs" pod "5fdc740e-7cf5-4b19-b7af-4cd2beed625c" (UID: "5fdc740e-7cf5-4b19-b7af-4cd2beed625c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:11:41.216051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.215995    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca" pod "5fdc740e-7cf5-4b19-b7af-4cd2beed625c" (UID: "5fdc740e-7cf5-4b19-b7af-4cd2beed625c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:11:41.216383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.216357    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg" pod "5fdc740e-7cf5-4b19-b7af-4cd2beed625c" (UID: "5fdc740e-7cf5-4b19-b7af-4cd2beed625c"). InnerVolumeSpecName "kube-api-access-rktxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:11:41.221344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.221317    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls" pod "5fdc740e-7cf5-4b19-b7af-4cd2beed625c" (UID: "5fdc740e-7cf5-4b19-b7af-4cd2beed625c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:11:41.230255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.230228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.312628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.312423    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:41.312628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.312492    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rktxg\" (UniqueName: \"kubernetes.io/projected/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-kube-api-access-rktxg\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:41.312628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.312510    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:41.312628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.312523    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fdc740e-7cf5-4b19-b7af-4cd2beed625c-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:41.367223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.367200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.627077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.626997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.780974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.780942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:41.865077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.865042    6738 generic.go:356] "Generic (PLEG): container finished" podID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" containerID="5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248" exitCode=0
Oct 30 21:11:41.865537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.865101    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"
Oct 30 21:11:41.865537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.865138    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" event={"ID":"5fdc740e-7cf5-4b19-b7af-4cd2beed625c","Type":"ContainerDied","Data":"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"}
Oct 30 21:11:41.865537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.865204    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln" event={"ID":"5fdc740e-7cf5-4b19-b7af-4cd2beed625c","Type":"ContainerDied","Data":"a77a1eeabbf57984083ad58f0903bb7a61a9745e03c883ef3b9bd7ebe54cc8af"}
Oct 30 21:11:41.865537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.865234    6738 scope.go:122] "RemoveContainer" containerID="5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"
Oct 30 21:11:41.875223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.875206    6738 scope.go:122] "RemoveContainer" containerID="5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"
Oct 30 21:11:41.875526 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:41.875504    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248\": container with ID starting with 5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248 not found: ID does not exist" containerID="5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"
Oct 30 21:11:41.875526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.875533    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248"} err="failed to get container status \"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248\": rpc error: code = NotFound desc = could not find container \"5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248\": container with ID starting with 5f8a52e747dc711519cc0c4c75caa608293dd3556a351f9a3918adf746c02248 not found: ID does not exist"
Oct 30 21:11:41.898644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.898593    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:41.909356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.909334    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e2cef583-6888cd64c6-jbhln"]
Oct 30 21:11:41.911816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:41.911788    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.127824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.127765    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.343953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.343863    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:42.344137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.344110    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" podUID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" containerName="maas-api" containerID="cri-o://b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb" gracePeriod=30
Oct 30 21:11:42.418713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.418666    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.649247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.649142    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.687137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.687107    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:42.776524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.776489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.839543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.839511    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\") pod \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") "
Oct 30 21:11:42.839814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.839614    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\" (UniqueName: \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\") pod \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") "
Oct 30 21:11:42.839814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.839692    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\") pod \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") "
Oct 30 21:11:42.839814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.839762    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\") pod \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\" (UID: \"04c24117-dbe8-4d5d-953c-7f480f5b68f4\") "
Oct 30 21:11:42.839995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.839913    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca" pod "04c24117-dbe8-4d5d-953c-7f480f5b68f4" (UID: "04c24117-dbe8-4d5d-953c-7f480f5b68f4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:11:42.840136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.840112    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04c24117-dbe8-4d5d-953c-7f480f5b68f4-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:42.842580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.842501    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls" pod "04c24117-dbe8-4d5d-953c-7f480f5b68f4" (UID: "04c24117-dbe8-4d5d-953c-7f480f5b68f4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:11:42.845032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.842673    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc" pod "04c24117-dbe8-4d5d-953c-7f480f5b68f4" (UID: "04c24117-dbe8-4d5d-953c-7f480f5b68f4"). InnerVolumeSpecName "kube-api-access-ccbbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:11:42.847950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.847922    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs" pod "04c24117-dbe8-4d5d-953c-7f480f5b68f4" (UID: "04c24117-dbe8-4d5d-953c-7f480f5b68f4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.855562    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856612    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" containerName="maas-api"
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856633    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" containerName="maas-api"
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856646    6738 cpu_manager.go:414] "containerMap: removing container" podUID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" containerName="maas-api"
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856654    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" containerName="maas-api"
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856773    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" containerName="maas-api"
Oct 30 21:11:42.860509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.856786    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" containerName="maas-api"
Oct 30 21:11:42.860941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.860853    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:42.874071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.873997    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:11:42.875706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.875647    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-05a342-serving-cert\""
Oct 30 21:11:42.877411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.877386    6738 generic.go:356] "Generic (PLEG): container finished" podID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" containerID="b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb" exitCode=0
Oct 30 21:11:42.877760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.877566    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" event={"ID":"04c24117-dbe8-4d5d-953c-7f480f5b68f4","Type":"ContainerDied","Data":"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"}
Oct 30 21:11:42.877760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.877643    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs" event={"ID":"04c24117-dbe8-4d5d-953c-7f480f5b68f4","Type":"ContainerDied","Data":"938d02b9903fd88749e72ca28a6d129ee427985f9c9296eaf31868ceec4d5ee9"}
Oct 30 21:11:42.877760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.877677    6738 scope.go:122] "RemoveContainer" containerID="b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"
Oct 30 21:11:42.877760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.877745    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"
Oct 30 21:11:42.905985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.901916    6738 scope.go:122] "RemoveContainer" containerID="b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"
Oct 30 21:11:42.905985 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:42.903988    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb\": container with ID starting with b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb not found: ID does not exist" containerID="b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"
Oct 30 21:11:42.905985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.904020    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb"} err="failed to get container status \"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb\": rpc error: code = NotFound desc = could not find container \"b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb\": container with ID starting with b8833052d5778142a32ffcd7946364e4fe5bb870b3c7ef25595ee043214e0acb not found: ID does not exist"
Oct 30 21:11:42.912173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.912138    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:42.926605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.926579    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:42.936308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.936274    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-111fd691-867cdcc74b-qwprs"]
Oct 30 21:11:42.940803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.940778    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccbbc\" (UniqueName: \"kubernetes.io/projected/04c24117-dbe8-4d5d-953c-7f480f5b68f4-kube-api-access-ccbbc\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:42.940898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.940823    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:42.940898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:42.940835    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04c24117-dbe8-4d5d-953c-7f480f5b68f4-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:11:43.039058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.038986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:43.042578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.042550    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j929q\" (UniqueName: \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.043056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.042992    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.043193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.043150    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.043365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.043347    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.144727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.144679    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.144727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.144731    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j929q\" (UniqueName: \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.144980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.144775    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.144980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.144808    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.145800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.145774    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.147538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.147511    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.147633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.147556    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.153209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.153190    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j929q\" (UniqueName: \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\") pod \"maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.167320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.167254    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04c24117-dbe8-4d5d-953c-7f480f5b68f4" path="/var/lib/kubelet/pods/04c24117-dbe8-4d5d-953c-7f480f5b68f4/volumes"
Oct 30 21:11:43.167846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.167826    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fdc740e-7cf5-4b19-b7af-4cd2beed625c" path="/var/lib/kubelet/pods/5fdc740e-7cf5-4b19-b7af-4cd2beed625c/volumes"
Oct 30 21:11:43.177748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.177724    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:43.215623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.215589    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:43.309054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.309017    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:43.494387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.494354    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:11:43.736314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.736284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:43.884782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:43.884747    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" event={"ID":"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb","Type":"ContainerStarted","Data":"eda96ee7a0ff0599c6e51d47272bad76265ecded7a6a0f7ddf1ff5106c46351a"}
Oct 30 21:11:44.073912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.073881    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.202732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.202653    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.546683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.546582    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.709025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.708991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.893617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.893578    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" event={"ID":"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb","Type":"ContainerStarted","Data":"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"}
Oct 30 21:11:44.894092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.893687    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:44.899683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.899658    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:44.918064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:44.918002    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" podStartSLOduration=2.47409709 podStartE2EDuration="2.917989116s" totalImagesPullingTime="443.892026ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:44.91662717 +0000 UTC m=+561.872314399" watchObservedRunningTime="2026-10-30 21:11:44.917989116 +0000 UTC m=+561.873676344"
Oct 30 21:11:45.066786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:45.066753    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:45.202880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:45.202801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:45.331845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:45.331817    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:46.133186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:46.133151    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:46.342411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:46.342378    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:46.474226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:46.474144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:46.606967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:46.606925    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.104476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:47.104326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.418571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:47.418537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.599292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:47.599251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.739311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:47.739220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:47.980969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:47.980931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.117472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.117417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.245075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.245039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.378945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.378857    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.501333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.501291    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.724756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.724654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:48.874311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:48.874268    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.014247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:49.014115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.227091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:49.227052    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.427300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:49.427259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.655039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:49.655003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:49.807700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:49.807619    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.053427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.053392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.647315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.647284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.912588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.912505    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:11:50.918085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.918061    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:11:50.918930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.918190    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:50.925448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.925407    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-3011c6-serving-cert\""
Oct 30 21:11:50.935269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.934579    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:11:50.942919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.942867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:50.943642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.943624    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:50.943796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.943778    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:50.943868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.943809    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7z5m\" (UniqueName: \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:50.943928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:50.943896    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.044542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.044497    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.044732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.044615    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.044732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.044642    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7z5m\" (UniqueName: \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.044732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.044705    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.045834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.045737    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.047249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.047222    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.047479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.047465    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.060888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.060866    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7z5m\" (UniqueName: \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\") pod \"maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.098798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.098772    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:51.250360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.248310    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:51.541849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.541819    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:11:51.544137 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:11:51.544108    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a990500_0296_4a9f_8fd0_7cf50282841e.slice/crio-331c9cb2debca2931140c3b6623e809f7e2b26c4de98be1ad4e0ac650a647648 WatchSource:0}: Error finding container 331c9cb2debca2931140c3b6623e809f7e2b26c4de98be1ad4e0ac650a647648: Status 404 returned error can't find the container with id 331c9cb2debca2931140c3b6623e809f7e2b26c4de98be1ad4e0ac650a647648
Oct 30 21:11:51.951126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:51.951097    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" event={"ID":"2a990500-0296-4a9f-8fd0-7cf50282841e","Type":"ContainerStarted","Data":"331c9cb2debca2931140c3b6623e809f7e2b26c4de98be1ad4e0ac650a647648"}
Oct 30 21:11:52.041222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.041175    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.169044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.169012    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.282271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.282187    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.529420 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:52.529382    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:11:52.546045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.545586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.678891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.678860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:52.957656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.957580    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" event={"ID":"2a990500-0296-4a9f-8fd0-7cf50282841e","Type":"ContainerStarted","Data":"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"}
Oct 30 21:11:52.957656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.957637    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:52.976796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:52.976734    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" podStartSLOduration=2.598529405 podStartE2EDuration="2.976715645s" totalImagesPullingTime="378.18624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:52.975004583 +0000 UTC m=+569.930691810" watchObservedRunningTime="2026-10-30 21:11:52.976715645 +0000 UTC m=+569.932402874"
Oct 30 21:11:53.669856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:53.669822    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.821040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:53.821003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:53.947066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:53.946974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:54.171493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:54.171460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:55.162634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:55.162599    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:55.991413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:55.991373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:56.740298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.740256    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:11:56.744966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.744943    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.757371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.757349    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-a33b12-serving-cert\""
Oct 30 21:11:56.761660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.761636    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:11:56.804707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.804679    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.805096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.805060    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.805228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.805175    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.805360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.805270    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csq5b\" (UniqueName: \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.822292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.822270    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:56.905985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.905962    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csq5b\" (UniqueName: \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.906087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.906017    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.906087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.906076    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.906184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.906131    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.906184 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:56.906165    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-serving-cert: secret "maas-api-e2e-worker-w1-a33b12-serving-cert" not found
Oct 30 21:11:56.906260 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:11:56.906251    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls podName:fc0446c6-73f2-43f5-9e6b-7a2f39126a8f nodeName:}" failed. No retries permitted until 2026-10-30 21:11:57.406229389 +0000 UTC m=+574.361916615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls") pod "maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" (UID: "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f") : secret "maas-api-e2e-worker-w1-a33b12-serving-cert" not found
Oct 30 21:11:56.906761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.906704    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.908682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.908661    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.917116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.917090    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csq5b\" (UniqueName: \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:56.945540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:56.945509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:57.135535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.135495    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:57.410932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.410849    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:57.413464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.413402    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") pod \"maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:57.621890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.621838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:57.660735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.660702    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:57.945047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.945026    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:11:57.947998 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:11:57.947969    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc0446c6_73f2_43f5_9e6b_7a2f39126a8f.slice/crio-82a766d62c6271dd1299043ecd080068b3aa0594bba43709ccf5e77ad459fa58 WatchSource:0}: Error finding container 82a766d62c6271dd1299043ecd080068b3aa0594bba43709ccf5e77ad459fa58: Status 404 returned error can't find the container with id 82a766d62c6271dd1299043ecd080068b3aa0594bba43709ccf5e77ad459fa58
Oct 30 21:11:57.985986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:57.985956    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" event={"ID":"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f","Type":"ContainerStarted","Data":"82a766d62c6271dd1299043ecd080068b3aa0594bba43709ccf5e77ad459fa58"}
Oct 30 21:11:58.416419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:58.416375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:11:58.970674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:58.970650    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:11:58.995533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:58.995272    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" event={"ID":"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f","Type":"ContainerStarted","Data":"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"}
Oct 30 21:11:58.995533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:58.995342    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:11:59.015479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:59.013502    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" podStartSLOduration=2.603505941 podStartE2EDuration="3.013483457s" totalImagesPullingTime="409.977516ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:11:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:11:59.011919898 +0000 UTC m=+575.967607127" watchObservedRunningTime="2026-10-30 21:11:59.013483457 +0000 UTC m=+575.969170685"
Oct 30 21:11:59.279885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:11:59.279802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.040811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:00.040779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.518084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:00.518048    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:00.920817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:00.920776    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.236824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:01.236746    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.351803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:01.351763    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.655352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:01.655320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:01.845946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:01.845907    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.013871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:02.013788    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.135936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:02.135898    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.259760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:02.259729    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:02.691031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:02.690968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:03.239620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:03.239584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:03.649082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:03.649046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:03.912950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:03.912870    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.238650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:04.238570    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.390424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:04.390381    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:04.803952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:04.803919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:05.006563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:05.006533    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:12:05.549710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:05.549673    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:06.259007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:06.258970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:07.130634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:07.130596    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:07.891909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:07.891872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:08.557503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:08.557466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:08.882246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:08.882205    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:09.198716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:09.198614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:09.327387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:09.327352    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:09.442638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:09.442605    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:09.753534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:09.753502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:10.250234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:10.250197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:10.545537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:10.545422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:10.894671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:10.894635    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.005879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.005847    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.344462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.344322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.584706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.584668    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:12:11.588854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.588830    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.600782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.600743    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:12:11.601952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.601160    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3011c6\"/\"default-dockercfg-wh42t\""
Oct 30 21:12:11.602084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.601713    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3011c6\"/\"simulator-w4-3011c6-kserve-self-signed-certs\""
Oct 30 21:12:11.610633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.610609    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3011c6\"/\"openshift-service-ca.crt\""
Oct 30 21:12:11.610759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.610710    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-3011c6\"/\"kube-root-ca.crt\""
Oct 30 21:12:11.658369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.658338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:11.666838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.666806    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.666966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.666865    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.667014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.666977    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.667065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.667035    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.667154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.667135    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82f49\" (UniqueName: \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.667214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.667187    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768395    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768448    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768474    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768502    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768530    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82f49\" (UniqueName: \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768569    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768874    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.768970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.768882    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.769110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.769063    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.771472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.771412    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.771716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.771683    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.776745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.776722    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82f49\" (UniqueName: \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\") pod \"simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") " pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:11.905337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:11.905295    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:12.191343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:12.191314    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:12:12.193515 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:12:12.193490    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09aeb068_df15_48d0_9ec4_17a6558620a0.slice/crio-a7b3f8f0d222bc1d3f197ce3b3281f46119a5dcdeaedf8bfec78207f5ab44812 WatchSource:0}: Error finding container a7b3f8f0d222bc1d3f197ce3b3281f46119a5dcdeaedf8bfec78207f5ab44812: Status 404 returned error can't find the container with id a7b3f8f0d222bc1d3f197ce3b3281f46119a5dcdeaedf8bfec78207f5ab44812
Oct 30 21:12:12.225413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:12.225376    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:12.584247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:12.584165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:13.075835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:13.075787    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" event={"ID":"09aeb068-df15-48d0-9ec4-17a6558620a0","Type":"ContainerStarted","Data":"a7b3f8f0d222bc1d3f197ce3b3281f46119a5dcdeaedf8bfec78207f5ab44812"}
Oct 30 21:12:13.183890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:13.183861    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:13.380124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:13.380079    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:13.516807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:13.516659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:14.082698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:14.082601    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" event={"ID":"09aeb068-df15-48d0-9ec4-17a6558620a0","Type":"ContainerStarted","Data":"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"}
Oct 30 21:12:14.103065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:14.103013    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podStartSLOduration=1.6289255329999999 podStartE2EDuration="3.103000094s" totalImagesPullingTime="1.474074561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:14.101710174 +0000 UTC m=+591.057397406" watchObservedRunningTime="2026-10-30 21:12:14.103000094 +0000 UTC m=+591.058687321"
Oct 30 21:12:14.336276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:14.336194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:14.905610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:14.905573    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:15.240226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:15.240146    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:16.145479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:16.145446    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.023997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:17.023961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.256701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:17.256654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:17.728345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:17.728309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:18.527170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:18.527137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.216097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:19.216061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.318375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:19.318329    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:19.911468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:19.911422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.201340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.201251    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:12:20.209993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.209964    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.214469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.214121    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:12:20.218259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.217622    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"openshift-service-ca.crt\""
Oct 30 21:12:20.218259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.217711    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"kube-root-ca.crt\""
Oct 30 21:12:20.218259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.217847    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"default-dockercfg-p666d\""
Oct 30 21:12:20.218259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.217923    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"simulator-w1-a33b12-kserve-self-signed-certs\""
Oct 30 21:12:20.271316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.271245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.358881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.358845    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.359045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.358898    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.359045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.358925    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.359118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.359052    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8bcq\" (UniqueName: \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.359177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.359156    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.359230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.359215    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.459911    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8bcq\" (UniqueName: \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460028    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460069    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460115    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460209    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460242    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460478    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460768    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.460914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.460810    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.462712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.462684    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.462940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.462924    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.468202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.468182    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8bcq\" (UniqueName: \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\") pod \"simulator-w1-a33b12-kserve-77f96b885d-8vk2h\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") " pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.534271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.534228    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:20.809246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.809122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:20.821165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:20.821138    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:12:20.824635 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:12:20.824592    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d97fa3d_1a83_4083_9368_84fcd10957f3.slice/crio-a39b46a1f7a7c2f2eb6e9dcc4b2a4296eef4e8280cf628688a6e7476baec0ddf WatchSource:0}: Error finding container a39b46a1f7a7c2f2eb6e9dcc4b2a4296eef4e8280cf628688a6e7476baec0ddf: Status 404 returned error can't find the container with id a39b46a1f7a7c2f2eb6e9dcc4b2a4296eef4e8280cf628688a6e7476baec0ddf
Oct 30 21:12:21.126820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.126633    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" event={"ID":"6d97fa3d-1a83-4083-9368-84fcd10957f3","Type":"ContainerStarted","Data":"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"}
Oct 30 21:12:21.126820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.126688    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" event={"ID":"6d97fa3d-1a83-4083-9368-84fcd10957f3","Type":"ContainerStarted","Data":"a39b46a1f7a7c2f2eb6e9dcc4b2a4296eef4e8280cf628688a6e7476baec0ddf"}
Oct 30 21:12:21.143562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.143503    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podStartSLOduration=1.143486419 podStartE2EDuration="1.143486419s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:21.142888701 +0000 UTC m=+598.098575928" watchObservedRunningTime="2026-10-30 21:12:21.143486419 +0000 UTC m=+598.099173646"
Oct 30 21:12:21.491531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.491452    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:21.534719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.534692    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:21.906033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.905996    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:21.924048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:21.924026    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:22.149839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:22.149805    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:12:22.217512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:22.217400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:22.510381 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:12:22.510295    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:12:23.018932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:23.018899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:23.727623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:23.727584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:24.437385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:24.437351    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:24.560916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:24.560876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:25.335695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:25.335656    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.099357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.099324    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:12:26.106127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.106100    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.110867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.110844    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:12:26.115749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.115621    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"default-dockercfg-5qtcc\""
Oct 30 21:12:26.115749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.115715    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"premium-w3-bfc50e-kserve-self-signed-certs\""
Oct 30 21:12:26.116140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.116092    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"openshift-service-ca.crt\""
Oct 30 21:12:26.116327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.116294    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"kube-root-ca.crt\""
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.121791    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.121874    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqfq9\" (UniqueName: \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.121940    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.121965    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.122032    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.122215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.122073    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.152097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.152056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.222756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222725    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfq9\" (UniqueName: \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.222937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222813    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.222937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222841    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.222937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222906    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.223095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222947    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.223095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.222988    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.223685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.223417    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.223685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.223489    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.223685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.223636    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.226775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.226752    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.242190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.242155    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.246490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.246259    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfq9\" (UniqueName: \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\") pod \"premium-w3-bfc50e-kserve-659f8f4d86-lmr4v\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") " pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.343736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.343694    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:26.429819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.429791    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:26.708341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:26.708312    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:12:26.710252 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:12:26.710223    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod753953a8_b9be_4e08_a259_6d990fde1708.slice/crio-8bc21c01383176dba3726887a29a28a523e8ba98736ce483f2de59140a62c837 WatchSource:0}: Error finding container 8bc21c01383176dba3726887a29a28a523e8ba98736ce483f2de59140a62c837: Status 404 returned error can't find the container with id 8bc21c01383176dba3726887a29a28a523e8ba98736ce483f2de59140a62c837
Oct 30 21:12:27.054905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:27.054814    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:27.164282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:27.164247    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" event={"ID":"753953a8-b9be-4e08-a259-6d990fde1708","Type":"ContainerStarted","Data":"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"}
Oct 30 21:12:27.164481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:27.164294    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" event={"ID":"753953a8-b9be-4e08-a259-6d990fde1708","Type":"ContainerStarted","Data":"8bc21c01383176dba3726887a29a28a523e8ba98736ce483f2de59140a62c837"}
Oct 30 21:12:27.430149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:27.430111    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:27.845869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:27.845784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:28.553017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:28.552986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:29.548903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:29.548865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:30.355794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:30.355760    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:30.534995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:30.534962    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:30.548313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:30.548290    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:30.568172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:30.568126    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podStartSLOduration=4.568113048 podStartE2EDuration="4.568113048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:27.182461337 +0000 UTC m=+604.138148556" watchObservedRunningTime="2026-10-30 21:12:30.568113048 +0000 UTC m=+607.523800275"
Oct 30 21:12:31.197854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:31.197823    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:12:31.277799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:31.277768    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.027610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:32.027578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:32.744847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:32.744796    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:33.544423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:33.544371    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:34.241986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:34.241955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:34.715700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:34.715664    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:35.023676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:35.023603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:35.742540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:35.742507    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.279179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.279146    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:12:36.289024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.289002    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.297074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.297042    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:12:36.298360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.298332    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-384164\"/\"default-dockercfg-mldjn\""
Oct 30 21:12:36.298719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.298699    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-384164\"/\"kube-root-ca.crt\""
Oct 30 21:12:36.299035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.299012    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-384164\"/\"premium-w0-384164-kserve-self-signed-certs\""
Oct 30 21:12:36.299144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.299131    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-384164\"/\"openshift-service-ca.crt\""
Oct 30 21:12:36.430866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.430839    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:36.432685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.432658    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.432913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.432887    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.433066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.433048    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.433144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.433082    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.433144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.433112    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nm98\" (UniqueName: \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.433345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.433266    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.435071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.435041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.447428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.447401    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:36.534052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.533979    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534025    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534063    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534138    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534163    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nm98\" (UniqueName: \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534471    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534774    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.534928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.534781    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.536878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.536855    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.537578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.537551    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.542918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.542890    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nm98\" (UniqueName: \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\") pod \"premium-w0-384164-kserve-d9fbf8b88-xz45x\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") " pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.576914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.576882    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:36.615588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.615558    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:36.901700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:36.901664    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:12:36.904230 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:12:36.904198    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod251b4c4c_cc07_47da_82bf_555c8eed0861.slice/crio-15b922ee12443876ffc23ce240e94917efddd0fdc3a46a6ef82140f504d33399 WatchSource:0}: Error finding container 15b922ee12443876ffc23ce240e94917efddd0fdc3a46a6ef82140f504d33399: Status 404 returned error can't find the container with id 15b922ee12443876ffc23ce240e94917efddd0fdc3a46a6ef82140f504d33399
Oct 30 21:12:37.232800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.232706    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" event={"ID":"251b4c4c-cc07-47da-82bf-555c8eed0861","Type":"ContainerStarted","Data":"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"}
Oct 30 21:12:37.232800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.232758    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" event={"ID":"251b4c4c-cc07-47da-82bf-555c8eed0861","Type":"ContainerStarted","Data":"15b922ee12443876ffc23ce240e94917efddd0fdc3a46a6ef82140f504d33399"}
Oct 30 21:12:37.234282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.234254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:37.244735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.244704    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:12:37.251719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.251668    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podStartSLOduration=1.251653225 podStartE2EDuration="1.251653225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:12:37.25068161 +0000 UTC m=+614.206368837" watchObservedRunningTime="2026-10-30 21:12:37.251653225 +0000 UTC m=+614.207340453"
Oct 30 21:12:37.616671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:37.616642    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:38.038073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:38.037994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:38.956898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:38.956867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:39.737324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:39.737290    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:40.766519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:40.766484    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:41.460977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:41.460943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:42.139330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:42.139301    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:42.942828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:42.942786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:43.737259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:43.737218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:44.417191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:44.417154    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:44.724821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:44.724736    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:45.518297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:45.518254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:46.456991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:46.456957    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:46.616801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:46.616770    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:46.629469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:46.629426    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:47.256655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:47.256628    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:47.302078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:47.302053    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:12:47.939511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:47.939480    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:48.855172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:48.855120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:49.854917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:49.854883    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:50.766010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:50.765925    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:51.652922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:51.652892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:52.351424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:52.351390    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:52.507689 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:12:52.507645    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:12:53.379065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:53.379029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:54.151657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:54.151626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:54.899923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:54.899887    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:55.624100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:55.624073    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:56.467371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:56.467344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:57.224907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:57.224870    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:58.081585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:58.081551    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:58.875411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:58.875373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:12:59.839228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:12:59.839192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:00.660406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:00.660368    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:01.459740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:01.459686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:02.236504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:02.236448    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:02.930984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:02.930954    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:03.619414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:03.619373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:04.280652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:04.280614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:05.117176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:05.117145    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:06.016383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:06.016342    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:06.841096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:06.841061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:07.581064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:07.581037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:07.700201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:07.700162    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:08.317949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:08.317915    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.066945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:09.066916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:09.659019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:09.658985    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:10.374657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:10.374623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.021234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:11.021196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:11.705009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:11.704974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:12.355455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:12.355389    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:13.042626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:13.042591    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:13.748533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:13.748502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:14.435710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:14.435667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:15.239561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:15.239524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:15.940022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:15.939980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:16.771083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:16.771048    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:17.334553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:17.334515    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:17.559679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:17.559644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:18.248791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:18.248750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:19.126877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:19.126844    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:19.962872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:19.962842    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:20.649764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:20.649735    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:21.341891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:21.341761    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.058097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:22.058069    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:22.511803 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:13:22.511763    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:13:22.739126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:22.739093    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:23.435744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:23.435692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:24.154277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:24.154230    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:24.847656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:24.847616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:25.541226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:25.541189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:26.249473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:26.249424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:27.054881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:27.054844    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:27.750019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:27.749980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:28.434510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:28.434479    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:28.850786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:28.850676    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:28.984096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:28.984061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:29.366368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:29.366333    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:29.491504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:29.491468    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:30.281372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:30.281334    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:31.242328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:31.242293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:32.515952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:32.515912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:33.205589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:33.205554    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:33.888694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:33.888657    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:34.828883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:34.828843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:35.523723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:35.523695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:36.353191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:36.353164    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:37.221178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:37.221133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:38.057457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.057398    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:38.179834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.179791    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:13:38.261643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.261615    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:13:38.262264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.262243    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.303459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.291566    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-05a342-openshift-default-dockercfg-vxlf4\""
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.409963    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410026    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410074    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410113    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410155    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410224    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410268    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410338    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.412336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.410368    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7kgr\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.510981    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511037    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511081    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511110    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511146    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511211    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511249    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511314    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.511336    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7kgr\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.512475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.512490    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.522462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.513737    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.523338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.523314    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.523730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.523677    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.540856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.533550    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.546257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.546201    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.547189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.547160    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.563469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.555592    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7kgr\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.573141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.573115    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.670697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.670614    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:38.810681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.810648    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:13:38.810930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:38.810901    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:13:39.082609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.082222    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:39.417551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.417477    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:13:39.424347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.424306    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:39.424563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.424540    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:39.424699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.424682    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:39.628799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.628754    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" event={"ID":"b708ac00-44da-416f-bb61-291ab5148f74","Type":"ContainerStarted","Data":"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"}
Oct 30 21:13:39.629019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.628824    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" event={"ID":"b708ac00-44da-416f-bb61-291ab5148f74","Type":"ContainerStarted","Data":"a4036ad40e2d21be07aacbac931b44e188fecb125f128191c2b855d4af781c49"}
Oct 30 21:13:39.652458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.650209    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podStartSLOduration=1.650191883 podStartE2EDuration="1.650191883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:39.649153009 +0000 UTC m=+676.604840241" watchObservedRunningTime="2026-10-30 21:13:39.650191883 +0000 UTC m=+676.605879115"
Oct 30 21:13:39.671750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.671672    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:39.827678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:39.827639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:40.578849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:40.578815    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:40.673667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:40.673635    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:40.673832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:40.673685    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:41.429843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:41.429808    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:41.671934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:41.671904    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:41.672329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:41.671951    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:42.246340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:42.246312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:42.672193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:42.672157    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:42.672592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:42.672206    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:42.958631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:42.958471    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:43.656734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:43.656703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:43.671891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:43.671857    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:43.672054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:43.671913    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:44.536080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:44.536043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:44.672130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:44.672093    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:44.672311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:44.672162    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:45.434748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:45.434711    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:45.672054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:45.672021    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 21:13:45.672480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:45.672081    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 21:13:46.259928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.259895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.527698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.527602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.677064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.677030    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:46.677604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.677585    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:46.678566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.678377    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:46.678802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:46.678784    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:13:47.237615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:47.237579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:48.279869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:48.279832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:49.295304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:49.295228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:49.438972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:49.438939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:50.231728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:50.231695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:50.934003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:50.933962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:52.030925 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:52.030897    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:52.511124 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:13:52.511082    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:13:52.940401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:52.940371    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:53.646954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:53.646919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:54.342838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:54.342805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:55.042756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:55.042702    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:55.739180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:55.739149    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:56.615887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:56.615853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:57.332354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.332322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:57.722649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.722371    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:13:57.728074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.728049    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.735570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.735547    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-bfc50e-openshift-default-dockercfg-g92dm\""
Oct 30 21:13:57.742276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.742248    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:13:57.775124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.775085    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.775428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.775403    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmhbf\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.775620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.775599    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.775772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.775755    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.776000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.775981    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.776176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.776159    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.776287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.776271    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.776499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.776448    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.776816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.776795    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.882471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.882424    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhbf\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.882753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.882729    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.882941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.882923    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.883078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883062    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.883628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883606    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.883787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883768    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.883942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883922    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.884068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.884052    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.884266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.884246    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.885148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.885129    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.885286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883508    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.885693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.885673    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.885832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.883504    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.889048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.886026    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.889048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.886129    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.897341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.897310    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.911871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.911816    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhbf\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:57.914117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:57.914036    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:58.079836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.079510    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:58.313635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.313529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:58.409785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.408149    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:13:58.411872 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:13:58.411839    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eaf560a_6693_42a5_bb53_0fa44d5c79bd.slice/crio-24db9435beb754a01a2480f19dcca9daa9da0b2b25ea416e7c9339ba4b7052ab WatchSource:0}: Error finding container 24db9435beb754a01a2480f19dcca9daa9da0b2b25ea416e7c9339ba4b7052ab: Status 404 returned error can't find the container with id 24db9435beb754a01a2480f19dcca9daa9da0b2b25ea416e7c9339ba4b7052ab
Oct 30 21:13:58.414839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.414802    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:58.414954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.414885    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:58.414954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.414935    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:13:58.809177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.808325    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" event={"ID":"4eaf560a-6693-42a5-bb53-0fa44d5c79bd","Type":"ContainerStarted","Data":"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"}
Oct 30 21:13:58.809177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.808390    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" event={"ID":"4eaf560a-6693-42a5-bb53-0fa44d5c79bd","Type":"ContainerStarted","Data":"24db9435beb754a01a2480f19dcca9daa9da0b2b25ea416e7c9339ba4b7052ab"}
Oct 30 21:13:58.833474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:58.829007    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podStartSLOduration=1.8289894979999999 podStartE2EDuration="1.828989498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:13:58.828537122 +0000 UTC m=+695.784224392" watchObservedRunningTime="2026-10-30 21:13:58.828989498 +0000 UTC m=+695.784676726"
Oct 30 21:13:59.088986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.084512    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:13:59.089260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.089241    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:13:59.089419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.089395    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:13:59.326518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.326484    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:13:59.332419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.332388    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.345849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.345773    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-3011c6\"/\"e2e-conflict-a-ad08360f-kserve-self-signed-certs\""
Oct 30 21:13:59.381088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.380921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.471857    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.471920    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.471957    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.472001    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.472031    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.473603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.472119    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft6js\" (UniqueName: \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.511461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.510740    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594001    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594050    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594114    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft6js\" (UniqueName: \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594244    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594282    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.596506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.594311    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.647490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.602248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.649455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.648847    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.649643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.649095    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.672522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.670590    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.702474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.690039    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:13:59.702474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.692860    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.702474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.693573    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft6js\" (UniqueName: \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\") pod \"e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") " pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.702474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.702273    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:13:59.730298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.730264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:13:59.809633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.809594    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:13:59.815468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.810264    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.830468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.830284    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-3011c6-openshift-default-dockercfg-m96xs\""
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914010    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914068    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914104    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914150    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914234    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kffs\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914352    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914414    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914469    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:13:59.929473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:13:59.914506    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017049    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017135    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017174    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017205    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017252    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017303    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kffs\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017411    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017482    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.017988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.017515    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.025657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.018399    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.045456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.041297    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.045456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.041595    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.045456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.042067    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.045456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.042537    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.055457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.051734    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.081194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.080792    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:00.081194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.080853    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:00.090873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.088023    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.090873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.088942    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kffs\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.109735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.109706    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.389029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.389000    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:00.928225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:00.907077    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:14:01.054213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.050023    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" event={"ID":"6a61a79d-065e-4968-b714-31de909769a5","Type":"ContainerStarted","Data":"dda4a1747a20692fee23b5082c72806c2c96f8944e8f163969bf56d8d732ff78"}
Oct 30 21:14:01.084627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.080376    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:01.090554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.086178    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:01.279055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.276937    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:14:01.289899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.289185    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:01.289899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.289270    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:01.289899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.289313    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:01.935561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:01.935517    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:02.068067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.068025    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" event={"ID":"6a61a79d-065e-4968-b714-31de909769a5","Type":"ContainerStarted","Data":"a7db915e6ee7f93eb6c5184af0b82904a20b04021edc312c9ebdd2b935e61d89"}
Oct 30 21:14:02.069859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.069829    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" event={"ID":"097310d5-1f77-46d6-bd14-dbfafc5d4019","Type":"ContainerStarted","Data":"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"}
Oct 30 21:14:02.069979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.069922    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" event={"ID":"097310d5-1f77-46d6-bd14-dbfafc5d4019","Type":"ContainerStarted","Data":"98feed5577713334648317bef8bd6abb93901626f300bdec9877da34c68ace87"}
Oct 30 21:14:02.080489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.080461    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:02.080602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.080529    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:02.088274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.088224    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" podStartSLOduration=3.088205168 podStartE2EDuration="3.088205168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:02.084997845 +0000 UTC m=+699.040685067" watchObservedRunningTime="2026-10-30 21:14:02.088205168 +0000 UTC m=+699.043892397"
Oct 30 21:14:02.118462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.113828    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podStartSLOduration=3.113809475 podStartE2EDuration="3.113809475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:13:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:02.10717442 +0000 UTC m=+699.062861644" watchObservedRunningTime="2026-10-30 21:14:02.113809475 +0000 UTC m=+699.069496704"
Oct 30 21:14:02.396463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.392909    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:02.414553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.411756    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:02.414553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.411826    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:02.704638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.704547    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:14:02.961862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:02.961710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:03.080801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:03.080767    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:03.080971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:03.080822    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:03.394462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:03.390830    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:03.394462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:03.390902    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:04.010463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:04.010322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:04.080774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:04.080738    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:04.080974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:04.080812    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:04.418186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:04.418151    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:04.418484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:04.418429    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:05.021462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.015723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:05.080647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.080616    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:05.080824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.080664    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:05.391869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.391822    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:05.392068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.391900    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:05.945627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:05.945585    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:06.080399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:06.080369    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Oct 30 21:14:06.080818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:06.080417    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Oct 30 21:14:06.395459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:06.393818    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:06.395459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:06.393884    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:07.056814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.056780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:07.091460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.089724    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:14:07.091460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.090958    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:14:07.103468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.101575    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:14:07.259967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.259926    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:14:07.264948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.264924    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.272786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.272766    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:14:07.274504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.274480    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"distinct-w3-bfc50e-kserve-self-signed-certs\""
Oct 30 21:14:07.338856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.338780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:07.393820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.393788    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Oct 30 21:14:07.394092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.394066    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452092    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmrhj\" (UniqueName: \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452199    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452259    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452322    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452367    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.461460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.452414    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.553261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.553234    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.553902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.553881    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.554372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554359    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.554515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554504    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.554622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554611    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmrhj\" (UniqueName: \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.555108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554704    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.555108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554728    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.555108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.554271    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.555108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.555069    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.556572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.556544    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.557009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.556993    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.566093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.566075    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmrhj\" (UniqueName: \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\") pod \"distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.577678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.577660    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:07.896663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.895165    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:14:07.899025 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:14:07.898994    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbf52a31_a847_4ee2_9fcd_9c0ecb274750.slice/crio-af8fa60e5935afe490d6d1efb22d815973158f89115221af1add0a89ff146fae WatchSource:0}: Error finding container af8fa60e5935afe490d6d1efb22d815973158f89115221af1add0a89ff146fae: Status 404 returned error can't find the container with id af8fa60e5935afe490d6d1efb22d815973158f89115221af1add0a89ff146fae
Oct 30 21:14:07.961566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:07.961540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:08.151628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:08.151539    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" event={"ID":"dbf52a31-a847-4ee2-9fcd-9c0ecb274750","Type":"ContainerStarted","Data":"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"}
Oct 30 21:14:08.151628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:08.151591    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" event={"ID":"dbf52a31-a847-4ee2-9fcd-9c0ecb274750","Type":"ContainerStarted","Data":"af8fa60e5935afe490d6d1efb22d815973158f89115221af1add0a89ff146fae"}
Oct 30 21:14:08.170360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:08.170309    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podStartSLOduration=1.170295424 podStartE2EDuration="1.170295424s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:08.168874109 +0000 UTC m=+705.124561338" watchObservedRunningTime="2026-10-30 21:14:08.170295424 +0000 UTC m=+705.125982651"
Oct 30 21:14:08.577926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:08.577834    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:09.230862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.230832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:09.434280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.434227    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:14:09.434499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.434306    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:09.668747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.668717    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:09.669120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.669097    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:09.670051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.670034    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:14:09.703411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.703374    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:14:09.718559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:09.718530    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:14:10.019871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:10.019785    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:10.178554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:10.178523    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:14:10.729370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:10.729335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:11.430219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:11.430171    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:12.125341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.125307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:12.769829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.769799    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:14:12.775484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.775459    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.790944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.790921    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:14:12.879510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879476    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879526    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879653    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879678    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879754    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879791    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.879883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879834    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.880040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879953    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.880040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.879994    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwcvj\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.884623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.884406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:12.981216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981179    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981282    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981369    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981420    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwcvj\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981494    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981524    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981593    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981617    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.981875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981843    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.982018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.981990    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.982176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.982152    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.982348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.982298    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.982519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.982492    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.984057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.984036    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.984282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.984262    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.989863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.989839    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:12.992902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:12.992872    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwcvj\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:13.092287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.092220    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:13.379095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.379068    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:14:13.380742 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:14:13.380715    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a4e5ca_e7f4_4ba9_a603_5dd0e1fa3e03.slice/crio-3a97d29420a0066c29a5213ec1d998ca238986c789aba4683b82271803ff9ed8 WatchSource:0}: Error finding container 3a97d29420a0066c29a5213ec1d998ca238986c789aba4683b82271803ff9ed8: Status 404 returned error can't find the container with id 3a97d29420a0066c29a5213ec1d998ca238986c789aba4683b82271803ff9ed8
Oct 30 21:14:13.383271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.383243    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:13.383338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.383304    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:13.383338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.383333    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:14:13.618630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:13.618597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:14.209460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:14.195893    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" event={"ID":"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03","Type":"ContainerStarted","Data":"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"}
Oct 30 21:14:14.209460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:14.195966    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" event={"ID":"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03","Type":"ContainerStarted","Data":"3a97d29420a0066c29a5213ec1d998ca238986c789aba4683b82271803ff9ed8"}
Oct 30 21:14:14.234511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:14.234458    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podStartSLOduration=2.23442515 podStartE2EDuration="2.23442515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:14.231032567 +0000 UTC m=+711.186719796" watchObservedRunningTime="2026-10-30 21:14:14.23442515 +0000 UTC m=+711.190112436"
Oct 30 21:14:14.308462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:14.302894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:15.102460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:15.100485    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:15.103085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:15.103061    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:15.103558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:15.103528    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:15.105530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:15.105367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:15.837897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:15.837861    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:16.094894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:16.094808    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:16.095147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:16.095115    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:16.995254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:16.995214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:17.104003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.103970    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:17.104271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.104242    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:17.151482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.150735    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:17.535456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.535309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:17.584096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.584064    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:17.857517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:17.854765    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:18.093948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:18.093914    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:18.094568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:18.094523    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:18.314023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:18.313980    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:14:18.656473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:18.655098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:19.099510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:19.098790    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:19.099510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:19.098850    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:19.970464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:19.958847    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:20.102826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:20.102789    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Oct 30 21:14:20.103393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:20.103361    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Oct 30 21:14:21.539324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:21.539284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.136692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.136648    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:14:22.136903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.136750    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:14:22.312904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.312870    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:22.315189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.315162    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:22.316578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.316529    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:14:22.355948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:22.355911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:22.513073 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:14:22.512989    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:14:23.238716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:23.238665    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:23.687669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:23.687631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:23.798751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:23.798715    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:24.740886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:24.740834    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:25.628931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:25.628884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:26.345664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:26.345626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:27.056762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:27.056720    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:27.743923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:27.743885    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:28.438811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:28.438777    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:29.117480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:29.117420    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:29.823574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:29.823537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:29.942101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:29.942062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:30.553384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:30.553350    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:31.557468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:31.557370    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:32.565572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:32.565519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:33.580154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:33.580120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.655900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.655789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.816495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.812953    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:14:34.818761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.818734    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.826787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.826106    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"distinct-w1-a33b12-kserve-self-signed-certs\""
Oct 30 21:14:34.832248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.832224    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:14:34.894792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.894762    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:34.956252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.956165    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.956534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.956510    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.956724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.956708    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.956833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.956819    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.956963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.956948    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:34.957101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:34.957086    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7x2b\" (UniqueName: \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058088    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058411    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058467    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058531    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058598    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7x2b\" (UniqueName: \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058709    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058863    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.058996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.058910    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.059341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.059320    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.061007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.060981    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.061134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.061116    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.067654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.067621    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7x2b\" (UniqueName: \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\") pod \"distinct-w1-a33b12-kserve-7846d889b6-m7gm2\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.138054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.138017    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:35.426653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.426621    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:14:35.427524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:35.427495    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:35.429170 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:14:35.429146    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3493f046_8d90_4b9d_80be_4560d733732b.slice/crio-6b34fdf436049f88cbe03b64e423a7c94ec3f0cc919df1ea8e3b33e9e0f2d72a WatchSource:0}: Error finding container 6b34fdf436049f88cbe03b64e423a7c94ec3f0cc919df1ea8e3b33e9e0f2d72a: Status 404 returned error can't find the container with id 6b34fdf436049f88cbe03b64e423a7c94ec3f0cc919df1ea8e3b33e9e0f2d72a
Oct 30 21:14:36.120318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:36.120278    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:36.439936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:36.439848    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" event={"ID":"3493f046-8d90-4b9d-80be-4560d733732b","Type":"ContainerStarted","Data":"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"}
Oct 30 21:14:36.439936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:36.439900    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" event={"ID":"3493f046-8d90-4b9d-80be-4560d733732b","Type":"ContainerStarted","Data":"6b34fdf436049f88cbe03b64e423a7c94ec3f0cc919df1ea8e3b33e9e0f2d72a"}
Oct 30 21:14:36.458909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:36.458860    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podStartSLOduration=2.45884663 podStartE2EDuration="2.45884663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:36.457012229 +0000 UTC m=+733.412699459" watchObservedRunningTime="2026-10-30 21:14:36.45884663 +0000 UTC m=+733.414533859"
Oct 30 21:14:36.912180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:36.912140    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:37.138544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.138515    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:37.546016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.545982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:37.649369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.649338    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:14:37.654339 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.654318    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.665707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.665686    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bfc50e\"/\"distinct-2-w3-bfc50e-kserve-self-signed-certs\""
Oct 30 21:14:37.669099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.669076    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:14:37.696665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.696626    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.728613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.728580    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.732545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.719010    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:37.732725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.732698    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.732844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.732753    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn7mm\" (UniqueName: \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.732844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.732809    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.732935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.732897    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.833607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.833518    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.833948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.833928    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.833985    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834017    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn7mm\" (UniqueName: \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834067    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834149    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834489    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834694    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.834898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.834881    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.842095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.842067    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.848819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.848796    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn7mm\" (UniqueName: \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.853041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.851946    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\") pod \"distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") " pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:37.971674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:37.971632    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:38.263411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.263386    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:14:38.265499 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:14:38.265469    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a8e6aa2_6089_43c4_a030_e0bef25f5ccc.slice/crio-f80179eb0965e52f48b8e45f921ee2948035deab0eaad15d4b51adf48dddde81 WatchSource:0}: Error finding container f80179eb0965e52f48b8e45f921ee2948035deab0eaad15d4b51adf48dddde81: Status 404 returned error can't find the container with id f80179eb0965e52f48b8e45f921ee2948035deab0eaad15d4b51adf48dddde81
Oct 30 21:14:38.320208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.320181    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:38.453118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.453079    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" event={"ID":"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc","Type":"ContainerStarted","Data":"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"}
Oct 30 21:14:38.453118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.453121    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" event={"ID":"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc","Type":"ContainerStarted","Data":"f80179eb0965e52f48b8e45f921ee2948035deab0eaad15d4b51adf48dddde81"}
Oct 30 21:14:38.469902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.469857    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podStartSLOduration=1.469842448 podStartE2EDuration="1.469842448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:14:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:14:38.468902952 +0000 UTC m=+735.424590180" watchObservedRunningTime="2026-10-30 21:14:38.469842448 +0000 UTC m=+735.425529676"
Oct 30 21:14:38.971892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:38.971864    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:39.116138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:39.116080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:39.824936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:39.824884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:40.542163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:40.542125    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:41.535480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:41.535424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:42.526739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:42.526701    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:43.652462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:43.648767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:44.748616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:44.748578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:45.138465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:45.138421    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:45.151958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:45.151934    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:45.432247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:45.432167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:45.564584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:45.564557    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:14:46.126967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:46.126931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:46.859992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:46.859958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:46.984809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:46.984773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:47.728735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:47.728703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:47.972134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:47.972095    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:47.985477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:47.985375    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:48.519283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:48.519249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:48.586409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:48.586381    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:14:49.304645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:49.304612    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:14:49.353724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:49.353687    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:49.478743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:49.478702    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:50.219891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:50.219834    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:51.033873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:51.033816    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:51.823164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:51.823128    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:52.514073 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:14:52.514032    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:14:52.651529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:52.651496    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:53.523701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:53.523672    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:54.355911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:54.355875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:55.055881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:55.055852    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:56.215460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:56.207240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:56.352372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:56.344890    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:57.319462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:57.317670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:58.039618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:58.039577    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:58.865429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:58.865394    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:59.016360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:59.016325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:14:59.548692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:14:59.548662    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.295299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.295262    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.299384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.299358    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:15:00.304817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.304792    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.308870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.308847    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:15:00.311402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.311384    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-a33b12\"/\"distinct-2-w1-a33b12-kserve-self-signed-certs\""
Oct 30 21:15:00.410486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.410405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:00.465773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465735    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.465954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465801    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.465954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465854    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.465954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465886    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.465954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465923    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.466126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.465959    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f784k\" (UniqueName: \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.567664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.567580    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.567664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.567623    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.567902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.567741    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f784k\" (UniqueName: \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.567980    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.568036    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.568090    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.568134    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.568150    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.568393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.568375    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.570636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.570610    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.571255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.571224    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.592098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.592069    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f784k\" (UniqueName: \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\") pod \"distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") " pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.630356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.630078    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:00.918659 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:00.918630    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f11ec74_8136_402e_aac6_d8b7b903336a.slice/crio-fc24787e44bdfc86ca7a5e09ef517efd151d28fe57d850a6edb0088bfb2659b7 WatchSource:0}: Error finding container fc24787e44bdfc86ca7a5e09ef517efd151d28fe57d850a6edb0088bfb2659b7: Status 404 returned error can't find the container with id fc24787e44bdfc86ca7a5e09ef517efd151d28fe57d850a6edb0088bfb2659b7
Oct 30 21:15:00.919134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:00.919113    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:15:01.062839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:01.062801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:01.667454 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:01.667395    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" event={"ID":"2f11ec74-8136-402e-aac6-d8b7b903336a","Type":"ContainerStarted","Data":"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"}
Oct 30 21:15:01.668019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:01.667466    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" event={"ID":"2f11ec74-8136-402e-aac6-d8b7b903336a","Type":"ContainerStarted","Data":"fc24787e44bdfc86ca7a5e09ef517efd151d28fe57d850a6edb0088bfb2659b7"}
Oct 30 21:15:01.688241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:01.688185    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podStartSLOduration=1.688167644 podStartE2EDuration="1.688167644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:01.687311179 +0000 UTC m=+758.642998407" watchObservedRunningTime="2026-10-30 21:15:01.688167644 +0000 UTC m=+758.643854874"
Oct 30 21:15:01.751109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:01.751084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:02.576844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:02.576808    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:02.631089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:02.631059    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:02.741932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:02.741895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:02.944235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:02.944202    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:03.743584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:03.743553    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.337867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.337830    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" podUID="6a61a79d-065e-4968-b714-31de909769a5" containerName="main" containerID="cri-o://a7db915e6ee7f93eb6c5184af0b82904a20b04021edc312c9ebdd2b935e61d89" gracePeriod=45
Oct 30 21:15:04.580168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.580135    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:15:04.585846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.585820    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.635459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.631922    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-a33b12-openshift-default-dockercfg-6bm8g\""
Oct 30 21:15:04.635960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.635876    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:15:04.680283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680250    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.680488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680468    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.680601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680586    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.680772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680759    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.680903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680890    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzg8g\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.680998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.680986    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.681142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.681130    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.681237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.681224    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.681365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.681352    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.707939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.707879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:04.720018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.719904    6738 generic.go:356] "Generic (PLEG): container finished" podID="6a61a79d-065e-4968-b714-31de909769a5" containerID="a7db915e6ee7f93eb6c5184af0b82904a20b04021edc312c9ebdd2b935e61d89" exitCode=0
Oct 30 21:15:04.720405 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.720344    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" event={"ID":"6a61a79d-065e-4968-b714-31de909769a5","Type":"ContainerDied","Data":"a7db915e6ee7f93eb6c5184af0b82904a20b04021edc312c9ebdd2b935e61d89"}
Oct 30 21:15:04.755914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.755883    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:15:04.782891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.782861    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.782918    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.782950    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783065    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783115    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783157    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\" (UniqueName: \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\") pod \"6a61a79d-065e-4968-b714-31de909769a5\" (UID: \"6a61a79d-065e-4968-b714-31de909769a5\") "
Oct 30 21:15:04.783427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783403    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783469    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783549    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783609    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783637    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783662    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783792    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783826    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzg8g\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.783985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.783857    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.784158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.784131    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.784328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.784270    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.785241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785159    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.785357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785314    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.785452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785355    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.785514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785452    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:04.785568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785507    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:04.785679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.785658    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:04.786527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.786499    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:15:04.787282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.787249    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.787503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.787478    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "kube-api-access-ft6js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:15:04.787658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.787637    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm" pod "6a61a79d-065e-4968-b714-31de909769a5" (UID: "6a61a79d-065e-4968-b714-31de909769a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:15:04.788297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.788277    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.792240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.792220    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.792865 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.792846    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzg8g\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:04.885002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.884975    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ft6js\" (UniqueName: \"kubernetes.io/projected/6a61a79d-065e-4968-b714-31de909769a5-kube-api-access-ft6js\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:04.885002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.884998    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6a61a79d-065e-4968-b714-31de909769a5-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:04.885157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.885009    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:04.885157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.885018    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:04.885157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.885027    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:04.885157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:04.885036    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6a61a79d-065e-4968-b714-31de909769a5-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:15:05.054111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.054039    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:05.348246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.348212    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:15:05.348796 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:05.348771    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6418f2c2_9e0f_432f_bc5f_7c4f9a2b93ca.slice/crio-a6fab5f07f62607b7c0ee74666374bd6cb7473dd2ae4774e9d2e9729308de03b WatchSource:0}: Error finding container a6fab5f07f62607b7c0ee74666374bd6cb7473dd2ae4774e9d2e9729308de03b: Status 404 returned error can't find the container with id a6fab5f07f62607b7c0ee74666374bd6cb7473dd2ae4774e9d2e9729308de03b
Oct 30 21:15:05.351427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.351394    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:05.351552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.351498    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:05.351552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.351530    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:05.491417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.491372    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:05.731083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.730474    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"
Oct 30 21:15:05.731083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.730560    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g" event={"ID":"6a61a79d-065e-4968-b714-31de909769a5","Type":"ContainerDied","Data":"dda4a1747a20692fee23b5082c72806c2c96f8944e8f163969bf56d8d732ff78"}
Oct 30 21:15:05.731083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.730643    6738 scope.go:122] "RemoveContainer" containerID="a7db915e6ee7f93eb6c5184af0b82904a20b04021edc312c9ebdd2b935e61d89"
Oct 30 21:15:05.738596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.738559    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" event={"ID":"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca","Type":"ContainerStarted","Data":"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"}
Oct 30 21:15:05.738746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.738622    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" event={"ID":"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca","Type":"ContainerStarted","Data":"a6fab5f07f62607b7c0ee74666374bd6cb7473dd2ae4774e9d2e9729308de03b"}
Oct 30 21:15:05.751838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.751794    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:15:05.756363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.756341    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/e2e-conflict-a-ad08360f-kserve-85674bd6fc-lls2g"]
Oct 30 21:15:05.778910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:05.778870    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podStartSLOduration=1.7788577829999999 podStartE2EDuration="1.778857783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:05.775638286 +0000 UTC m=+762.731325547" watchObservedRunningTime="2026-10-30 21:15:05.778857783 +0000 UTC m=+762.734545011"
Oct 30 21:15:06.054592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:06.054505    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:06.056194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:06.056167    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:06.056315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:06.056222    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:06.243194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:06.243158    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:07.055158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:07.055114    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:07.055594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:07.055174    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:07.074836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:07.074800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:07.164752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:07.164716    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a61a79d-065e-4968-b714-31de909769a5" path="/var/lib/kubelet/pods/6a61a79d-065e-4968-b714-31de909769a5/volumes"
Oct 30 21:15:08.054964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.054925    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:08.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.055004    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:08.084192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.084153    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:08.440522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.297413    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:15:08.441295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.441272    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6a61a79d-065e-4968-b714-31de909769a5" containerName="main"
Oct 30 21:15:08.441397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.441300    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a61a79d-065e-4968-b714-31de909769a5" containerName="main"
Oct 30 21:15:08.441529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.441514    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a61a79d-065e-4968-b714-31de909769a5" containerName="main"
Oct 30 21:15:08.445664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.445638    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:15:08.445804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.445753    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.639134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.639093    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.639553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.639526    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.639683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.639670    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.639869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.639852    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.639983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.639969    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.640138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.640123    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.640231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.640219    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgplq\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.640362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.640348    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.640504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.640488    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.744225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744124    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.744488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744464    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.744700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744683    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.744820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744805    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgplq\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.744950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744935    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.745071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745058    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.745188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745173    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.745392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745376    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.745525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745508    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.753461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745584    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.753655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.746160    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.753923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.744860    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.794955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.745064    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.795126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.761885    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.795213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.773073    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.795311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.784638    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgplq\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.795403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.794038    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:08.795518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:08.794494    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:09.057914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.057818    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:09.058647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.058215    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:09.062648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.062622    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:09.718029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.717375    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:09.718029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.717501    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:09.718029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.717551    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:09.787859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.785840    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:15:09.863429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.863390    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" event={"ID":"9208f5f1-f275-4f16-85da-27adcad05ac4","Type":"ContainerStarted","Data":"4ff5c4b76e882abb52733b62f1696e009d3d3e1f905004b6fe091f6fe86a9df4"}
Oct 30 21:15:09.963742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:09.963707    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:10.056877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.056832    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:10.057162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.057127    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:10.635785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.635746    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:10.672663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.672632    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:10.870660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.870634    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" event={"ID":"9208f5f1-f275-4f16-85da-27adcad05ac4","Type":"ContainerStarted","Data":"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"}
Oct 30 21:15:10.882246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.882225    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:10.900521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.894369    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podStartSLOduration=2.894352699 podStartE2EDuration="2.894352699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:10.89280696 +0000 UTC m=+767.848494214" watchObservedRunningTime="2026-10-30 21:15:10.894352699 +0000 UTC m=+767.850039927"
Oct 30 21:15:10.962305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:10.962269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:11.058010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.056859    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:11.058010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.056938    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:11.070325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.070259    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:11.071856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.071829    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:11.072060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.071876    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:11.842892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:11.842859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:12.055274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:12.055242    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 21:15:12.055750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:12.055292    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 21:15:12.063611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:12.063584    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:12.063752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:12.063639    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:13.033088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:13.033035    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:13.063511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:13.063466    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:13.064110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:13.063514    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:14.063854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.063814    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:14.064362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.063875    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:14.112461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.111533    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:14.112461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.111597    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:14.140812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.140588    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:14.454814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.454784    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:14.455681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.455651    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:14.456294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.456270    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:15:14.969382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:14.969344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:15.081893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:15.081859    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:15.082506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:15.082474    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:15.100612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:15.100584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:15.213462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:15.212992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:16.063995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:16.063957    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 21:15:16.064197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:16.064017    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 21:15:16.425367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:16.425135    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:17.504413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:17.504378    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:17.576962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:17.576935    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:17.577327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:17.577303    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:17.578392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:17.578372    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:15:17.713976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:17.713944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:18.594071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:18.593846    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:19.228803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:19.228767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:19.709987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:19.709953    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:20.557621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:20.557576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:22.728554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:22.728512    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:15:22.786157 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:15:22.786119    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:15:22.882461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:22.881538    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:15:22.908730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:22.905873    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:23.145471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.143003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:23.153460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.149720    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:15:23.254472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.252455    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 21:15:23.772593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.772555    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:23.773159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.773128    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:23.822578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.822528    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:23.822835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:23.822810    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:24.493729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:24.493684    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:25.296919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:25.296873    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:26.190286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:26.190257    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:26.950668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:26.950615    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:27.733710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:27.733674    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:27.886760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:27.886728    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:15:27.892802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:27.892775    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:27.907581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:27.907556    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:15:28.011163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011069    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011125    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011259    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011285    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011320    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011430    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011510    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011571    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xps8w\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.011669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.011592    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112619    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112670    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112761    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112787    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112825    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112866    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.112967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112894    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.112977    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xps8w\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.113012    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.113214    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.113410    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.113517    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.113825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.113748    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.114118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.114095    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.115534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.115493    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.115822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.115794    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.121470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.121426    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xps8w\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.121555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.121468    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\") pod \"e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") " pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.221365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.221333    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:28.508355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.508304    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:15:28.512749 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:28.512721    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3638232_08bf_420e_b13b_12cb6dce0fd7.slice/crio-348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96 WatchSource:0}: Error finding container 348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96: Status 404 returned error can't find the container with id 348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96
Oct 30 21:15:28.515704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.515664    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:28.515783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.515751    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:28.515832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.515791    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:28.615293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:28.615260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:29.341579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.339825    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:29.355322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.351681    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" event={"ID":"a3638232-08bf-420e-b13b-12cb6dce0fd7","Type":"ContainerStarted","Data":"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"}
Oct 30 21:15:29.355322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.351751    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" event={"ID":"a3638232-08bf-420e-b13b-12cb6dce0fd7","Type":"ContainerStarted","Data":"348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96"}
Oct 30 21:15:29.754337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.754261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:29.847690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.847621    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podStartSLOduration=2.847591332 podStartE2EDuration="2.847591332s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:29.381858812 +0000 UTC m=+786.337546040" watchObservedRunningTime="2026-10-30 21:15:29.847591332 +0000 UTC m=+786.803278572"
Oct 30 21:15:29.848271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.848248    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:15:29.872232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.872199    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.875212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.875036    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:15:29.965963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.965928    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966040    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966101    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966122    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bwwk\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966150    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966263    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966303    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966382    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:29.966507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:29.966484    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067327    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067385    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067419    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067491    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067539    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067558    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bwwk\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067580    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067613    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.067714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067647    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.068100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.067926    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.068278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.068248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.068420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.068342    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.068525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.068420    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.068525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.068505    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.069986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.069966    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.070117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.070099    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.077185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.077166    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.079856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.079839    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bwwk\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.220106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.219633    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:30.221724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.221697    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:30.225092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.224909    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:30.225092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.224947    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:30.528549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.523530    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:15:30.529277 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:30.529252    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a870aa1_79b0_40e2_abf8_8e2c48176f6e.slice/crio-0682d9f555da66bc7404f9577928328a6d5b854b567ea7e94e345767ad63aff2 WatchSource:0}: Error finding container 0682d9f555da66bc7404f9577928328a6d5b854b567ea7e94e345767ad63aff2: Status 404 returned error can't find the container with id 0682d9f555da66bc7404f9577928328a6d5b854b567ea7e94e345767ad63aff2
Oct 30 21:15:30.532975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.532937    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:30.533085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.533025    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:30.533085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.533073    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:30.837584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:30.837498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:31.223878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:31.223842    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:31.224184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:31.224140    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:31.372271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:31.372242    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" event={"ID":"5a870aa1-79b0-40e2-abf8-8e2c48176f6e","Type":"ContainerStarted","Data":"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"}
Oct 30 21:15:31.372531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:31.372510    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" event={"ID":"5a870aa1-79b0-40e2-abf8-8e2c48176f6e","Type":"ContainerStarted","Data":"0682d9f555da66bc7404f9577928328a6d5b854b567ea7e94e345767ad63aff2"}
Oct 30 21:15:31.799012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:31.798974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:32.223475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.223036    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:32.223475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.223160    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:32.223475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.223203    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:32.235464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.232569    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:32.235464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.232631    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:32.753242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:32.753209    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:33.232465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:33.228802    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:33.232465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:33.228861    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:33.232465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:33.229459    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:33.232465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:33.229501    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:33.675574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:33.675534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:34.229463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.229075    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:34.229463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.229140    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:34.229757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.229520    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:34.229757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.229577    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:34.638458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.637959    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podStartSLOduration=5.637940193 podStartE2EDuration="5.637940193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:31.395681812 +0000 UTC m=+788.351369043" watchObservedRunningTime="2026-10-30 21:15:34.637940193 +0000 UTC m=+791.593627422"
Oct 30 21:15:34.638973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.638543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:34.641715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.640796    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:15:34.645762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.645739    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.674937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.674896    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:15:34.743820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.743764    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.743874    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.743920    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.743949    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.743973    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvh8q\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.744045    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.744072    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.744118    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.744225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.744161    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845571    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845613    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845631    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvh8q\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845690    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845709    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845742    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.845792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845774    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845814    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.845875    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.846085    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.846158    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.846341    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.846386    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.846929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.846776    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.848761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.848742    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.849080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.849066    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.854967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.854671    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.854967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.854910    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvh8q\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:34.979870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:34.979605    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:35.236911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.227022    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:35.236911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.227097    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:35.236911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.227527    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 21:15:35.236911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.227560    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 21:15:35.340043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.338949    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:15:35.351129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.351093    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:35.351245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.351172    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:35.351311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.351246    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:35.449461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.448409    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" event={"ID":"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6","Type":"ContainerStarted","Data":"8b983aa060e192fa796dfe6a87cef5ba1cc75ba1a69fcc1b29da71b1cdc84a48"}
Oct 30 21:15:35.861357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:35.861317    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:36.223519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.222796    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:36.223519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.222854    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:36.459308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.459279    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" event={"ID":"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6","Type":"ContainerStarted","Data":"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"}
Oct 30 21:15:36.481504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.481399    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podStartSLOduration=2.481386467 podStartE2EDuration="2.481386467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:36.480118694 +0000 UTC m=+793.435805923" watchObservedRunningTime="2026-10-30 21:15:36.481386467 +0000 UTC m=+793.437073694"
Oct 30 21:15:36.594456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.584305    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:36.594456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.585012    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:36.594456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.585857    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:15:36.834520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.833889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:36.980353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.980313    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:36.982158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.982131    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:36.982282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:36.982179    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:37.225460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:37.222567    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Oct 30 21:15:37.225460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:37.222630    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Oct 30 21:15:37.853997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:37.853955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:38.014458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.007489    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:38.014458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.007546    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:38.236458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.235796    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:38.237295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.237106    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:38.238690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.238672    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:15:38.455808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.455755    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:15:38.463536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.463511    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:15:38.463866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.463791    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.576937    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.576981    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjx9c\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577105    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577251    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577332    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577386    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577412    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577483    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.577560 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.577511    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.635580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.635549    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:15:38.646936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.646906    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:15:38.647115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.647026    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.653827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.653797    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-384164\"/\"distinct-2-w0-384164-kserve-self-signed-certs\""
Oct 30 21:15:38.679704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.679666    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.679866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.679721    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjx9c\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.679866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.679828    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.679988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.679930    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.679988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.679981    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.680106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.680017    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.680106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.680043    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.680106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.680082    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.680263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.680108    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.681413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.681382    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.681557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.681469    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.681557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.681528    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.681675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.681591    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.681823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.681805    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.684237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.684194    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.686131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.686113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.703960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.703890    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.705252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.705195    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjx9c\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\") pod \"e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") " pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.713873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.713789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:38.782189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782111    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.782375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782206    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.782375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782339    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.782544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782391    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gld2g\" (UniqueName: \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.782544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782471    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.782640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.782556    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.787798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.786641    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.884960    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.885041    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.885091    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.885234    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.885290    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gld2g\" (UniqueName: \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.887508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.885350    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.889854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.888903    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.889854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.889534    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.889854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.889803    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.904697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.897282    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gld2g\" (UniqueName: \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.918765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.912083    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.918765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.918718    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\") pod \"distinct-2-w0-384164-kserve-7756874f96-jjprv\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") " pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:38.984567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.984532    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:39.002404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.997185    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:39.002404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:38.997240    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:39.225464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.218549    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:15:39.245097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.245053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:39.247714 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:39.247646    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bfc79d6_0e95_42ba_a18d_9d3b2e8b8074.slice/crio-9e939ef5bb5fb74aa39250b290bb98dee2f52d3e2b618352f797fe92fe970ab8 WatchSource:0}: Error finding container 9e939ef5bb5fb74aa39250b290bb98dee2f52d3e2b618352f797fe92fe970ab8: Status 404 returned error can't find the container with id 9e939ef5bb5fb74aa39250b290bb98dee2f52d3e2b618352f797fe92fe970ab8
Oct 30 21:15:39.261730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.261676    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:39.261863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.261772    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:39.261863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.261820    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:39.448746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.448715    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:15:39.449187 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:39.449156    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2b80643_e89e_4ade_9e58_f4bf4d86749c.slice/crio-c71d354cb87d73356b069056eacaff57b359ca962cfb514399315b94cd449b49 WatchSource:0}: Error finding container c71d354cb87d73356b069056eacaff57b359ca962cfb514399315b94cd449b49: Status 404 returned error can't find the container with id c71d354cb87d73356b069056eacaff57b359ca962cfb514399315b94cd449b49
Oct 30 21:15:39.485156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.485128    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" event={"ID":"a2b80643-e89e-4ade-9e58-f4bf4d86749c","Type":"ContainerStarted","Data":"c71d354cb87d73356b069056eacaff57b359ca962cfb514399315b94cd449b49"}
Oct 30 21:15:39.497464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.495564    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" event={"ID":"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074","Type":"ContainerStarted","Data":"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"}
Oct 30 21:15:39.497464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.495623    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" event={"ID":"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074","Type":"ContainerStarted","Data":"9e939ef5bb5fb74aa39250b290bb98dee2f52d3e2b618352f797fe92fe970ab8"}
Oct 30 21:15:39.518460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.518187    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podStartSLOduration=1.5181676739999999 podStartE2EDuration="1.518167674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:39.517689832 +0000 UTC m=+796.473377062" watchObservedRunningTime="2026-10-30 21:15:39.518167674 +0000 UTC m=+796.473854903"
Oct 30 21:15:39.805609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.805531    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:39.985797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.980207    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:39.985797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:39.980285    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:40.502622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.501912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:40.571614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.571578    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" event={"ID":"a2b80643-e89e-4ade-9e58-f4bf4d86749c","Type":"ContainerStarted","Data":"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"}
Oct 30 21:15:40.795618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.793513    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:40.795618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.793588    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:40.982490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.980650    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:40.982490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:40.980715    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:41.003193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:41.003137    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:41.788462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:41.787573    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:41.788462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:41.787644    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:41.981843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:41.981808    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:41.982079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:41.982055    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:42.136462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.135811    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:15:42.241458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.238539    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.256467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.255478    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.297470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.295513    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:15:42.803891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.803848    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:42.804501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.804465    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:42.910598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.910545    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:42.990234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.990193    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:42.990574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:42.990538    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:43.140461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.138537    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.140461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.140285    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:15:43.300585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.243538    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.302158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.301000    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.453868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.453755    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podStartSLOduration=5.453733852 podStartE2EDuration="5.453733852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:40.603819243 +0000 UTC m=+797.559506457" watchObservedRunningTime="2026-10-30 21:15:43.453733852 +0000 UTC m=+800.409421080"
Oct 30 21:15:43.583980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.583936    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:15:43.652193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.652151    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.725524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.719536    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:43.824463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.802312    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:43.824463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.802373    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:43.894075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.893670    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.031541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.916528    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:15:44.039461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:43.999593    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:44.284521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.241547    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:44.323466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.177806    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.323466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.045535    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:15:44.324562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.324522    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:15:44.339840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.274205    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.341282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.306638    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.342187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.306576    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.632461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.628371    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:15:44.632461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.630084    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:15:44.672461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.665515    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:15:44.672461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.665971    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:15:44.703982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.701471    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:44.732463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.728892    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:44.846459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.835230    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:44.846459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.835306    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:44.898516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.895815    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:15:44.916461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:44.914678    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:45.093582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.087114    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:45.093882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.093844    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:45.130397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.110893    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:15:45.331262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.331159    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:15:45.341173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.341130    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 21:15:45.346176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.346138    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.350554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.350519    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.478460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.462228    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:15:45.692039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.691984    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.751797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.751751    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:45.787859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.787821    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:45.788109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.788079    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:45.899748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:45.899684    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.028971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.028844    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Oct 30 21:15:46.038054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.029210    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Oct 30 21:15:46.137240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.136513    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:15:46.185873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.185538    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.345940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.345840    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.466241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.346525    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:15:46.467634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.363552    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:15:46.477569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.413547    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.724463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.719575    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:46.803871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.803834    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:46.804139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.804110    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:46.813634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.813592    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:15:46.922463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:46.900527    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.142391    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.143110    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:15:47.440689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.440595    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.490020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.489970    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.514495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.490614    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.620601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.618882    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.620601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.619204    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:15:47.667460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.659880    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.720727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.720638    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.820052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.818265    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:47.831583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.824565    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:47.831583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.824635    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:47.831583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.825001    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:15:47.920458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.900535    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:15:47.920458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:47.901673    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.009534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.004065    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:15:48.062702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.062655    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:15:48.062989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.062964    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:15:48.140461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.097252    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:15:48.140461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.137264    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:15:48.161884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.160809    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:15:48.216035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.211168    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:48.217876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.217349    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:48.230097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.230066    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:15:48.234545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.230239    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:48.238474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.237857    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:15:48.238474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.237964    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:15:48.788915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.788882    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Oct 30 21:15:48.789205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.789175    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Oct 30 21:15:48.998659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:48.998604    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:49.011982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.011953    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:49.293692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.293660    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:15:49.361154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.357906    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:49.649933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.649896    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:15:49.655536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.655511    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.673544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.673514    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:15:49.728809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.728779    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.728873    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.728911    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.728970    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.729038    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c75mm\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.729076    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.729183    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.729221    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.729348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.729247    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830201    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830257    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830302    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830345    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c75mm\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830373    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830474    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830498    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830513    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830569    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.830908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.830892    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.831082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.831065    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.831754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.831735    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.832103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.832086    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.832639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.832608    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.834621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.834599    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.834728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.834703    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.872584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.872562    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.873002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.872977    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c75mm\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\") pod \"e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") " pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:49.972861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:49.972786    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:50.168243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.168198    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:50.168882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.168859    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:50.169587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.169565    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:15:50.276653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.276631    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:15:50.278601 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:15:50.278568    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94af8791_9541_42c1_98ea_7e3992bb4ecf.slice/crio-11edad2935567b2a218164066e9272a00776390a3cc919a9ec2b6ab8e6c8a5f7 WatchSource:0}: Error finding container 11edad2935567b2a218164066e9272a00776390a3cc919a9ec2b6ab8e6c8a5f7: Status 404 returned error can't find the container with id 11edad2935567b2a218164066e9272a00776390a3cc919a9ec2b6ab8e6c8a5f7
Oct 30 21:15:50.281268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.281235    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:50.281390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.281299    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:50.281390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.281328    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:15:50.293685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.293661    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" event={"ID":"94af8791-9541-42c1-98ea-7e3992bb4ecf","Type":"ContainerStarted","Data":"11edad2935567b2a218164066e9272a00776390a3cc919a9ec2b6ab8e6c8a5f7"}
Oct 30 21:15:50.526222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:50.526134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:51.344918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.344876    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" event={"ID":"94af8791-9541-42c1-98ea-7e3992bb4ecf","Type":"ContainerStarted","Data":"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"}
Oct 30 21:15:51.369962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.369908    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podStartSLOduration=2.36989078 podStartE2EDuration="2.36989078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:15:51.368700724 +0000 UTC m=+808.324387970" watchObservedRunningTime="2026-10-30 21:15:51.36989078 +0000 UTC m=+808.325578009"
Oct 30 21:15:51.705460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.703041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:51.974018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.973946    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:51.975420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.975394    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:51.975551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:51.975469    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:52.534373 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:15:52.534331    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:15:52.620571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:52.620538    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:52.973782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:52.973741    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:52.973946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:52.973798    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:53.477727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:53.477685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:53.973702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:53.973668    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:53.974106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:53.973720    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:54.348815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:54.348703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:54.974308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:54.974280    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:54.974702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:54.974332    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:55.370402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:55.370364    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:55.973975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:55.973943    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:55.974134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:55.973994    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:56.228342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:56.228308    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:56.973862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:56.973825    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 21:15:56.974070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:56.973880    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 21:15:57.124096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:57.124053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:57.930259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:57.930220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:57.979627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:57.979599    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:57.980566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:57.980541    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:57.981222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:57.981197    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:15:58.637371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:58.637338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:15:59.432011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:15:59.431976    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:00.498246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.498209    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:16:00.524155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.523009    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:16:00.524155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.523133    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.534168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.534136    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-384164-openshift-default-dockercfg-qs2n4\""
Oct 30 21:16:00.691833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.691004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:00.715822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.715780    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.717347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.717320    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.717588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.717566    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.717768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.717751    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.717972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.717954    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9xrq\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.718226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.718209    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.718373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.718358    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.723069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.723048    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.735849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.735826    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848033    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848120    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848186    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848283    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9xrq\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848339    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.848683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.848398    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.850283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.849170    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.850283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.849225    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.850283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.849293    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.850283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.849819    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.850283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.850109    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.851025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.850719    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.851335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.851314    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.851714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.851694    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.881561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.880709    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9xrq\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:00.916481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:00.915989    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:01.021905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:01.021805    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:01.027182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:01.027099    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:01.060054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:01.059739    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:01.687608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:01.687564    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:01.930617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:01.919140    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.116513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.115013    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.197461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.197161    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.316462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.315585    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:16:02.458082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.456019    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:02.458082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.456116    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:02.458082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.456163    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:02.503575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.502248    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.666536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.666393    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" event={"ID":"a1871633-1f97-451f-a9c5-8b414001e3a1","Type":"ContainerStarted","Data":"b1e0d5457fcf3e2460a5bd9dcffc35e9bdaefbc427cd58b27a650d8020636aae"}
Oct 30 21:16:02.735665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.735619    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:02.736176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.735689    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.760801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.737039    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:02.781774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.781613    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.895464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.876272    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:02.951518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.922410    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:16:02.953747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.953714    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:02.954657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:02.942883    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": context deadline exceeded"
Oct 30 21:16:03.125068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.121893    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.125068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.124153    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:16:03.200463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.199694    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.204502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.202570    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:16:03.286512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.286428    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.510539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.510336    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.513126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.511399    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:03.584461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.581859    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.709464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.677717    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:03.709464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.677778    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.803517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.793792    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.803517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.794924    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:03.804166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.804136    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:03.930498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.901402    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:16:03.948604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.913264    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:03.948857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.948831    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" event={"ID":"a1871633-1f97-451f-a9c5-8b414001e3a1","Type":"ContainerStarted","Data":"f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980"}
Oct 30 21:16:03.949565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.949540    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:03.949759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:03.949744    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:04.039454 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.037814    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:04.209571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.204551    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:04.209571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.207276    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:16:04.210123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.210101    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:04.242460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.216259    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:04.398825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.398726    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:04.589420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.589307    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:04.790812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.790752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:04.825531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.825503    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:16:04.828774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.828755    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:04.889589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.889545    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:04.978861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:04.978822    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:16:05.020427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.020351    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podStartSLOduration=5.020328597 podStartE2EDuration="5.020328597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:04.345888284 +0000 UTC m=+821.301575514" watchObservedRunningTime="2026-10-30 21:16:05.020328597 +0000 UTC m=+821.976015826"
Oct 30 21:16:05.078502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.078470    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:05.078843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.078810    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:05.082318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.082296    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:05.273153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.273061    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:05.710150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:05.710117    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:06.061171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:06.061100    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:06.061171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:06.061152    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:06.639066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:06.639010    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:07.061208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:07.061133    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:07.061208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:07.061188    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:07.565724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:07.565686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:08.061137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:08.061102    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:08.061314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:08.061153    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:08.443678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:08.443644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:09.060852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:09.060813    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:09.061047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:09.060867    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:09.356031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:09.355955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:10.060594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:10.060561    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:10.060763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:10.060612    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:10.201719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:10.201665    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:11.061351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:11.061320    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 21:16:11.061746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:11.061374    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 21:16:11.247538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:11.247503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:12.762465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:12.761811    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:12.771473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:12.763358    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:12.771473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:12.765648    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:16:12.937465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:12.933520    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:13.058721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.058592    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:13.071565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.071525    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:16:13.144519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.081520    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:13.218669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.218626    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:13.297462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.294705    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:13.494583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.494531    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:13.849520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:13.846918    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:16:14.040598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.040564    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:14.040878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.040850    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.067787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.067729    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.107643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.072510    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.109148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.108607    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:14.109411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.109397    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:14.242513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.242474    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.243147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.242478    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.243755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.243728    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:14.370569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.343875    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:16:14.370569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.345477    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.370569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.345777    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:14.505491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.495796    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:14.695029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.694938    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:14.719462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.706250    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:14.791139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.791114    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:14.890456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.868327    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:14.890456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.868757    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:14.890456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.872540    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:14.897890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:14.897865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:15.067764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.067680    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:16:15.077792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.075818    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.084906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.084872    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:16:15.138218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.138158    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.138510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.138483    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.138813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.138790    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.138961    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.139175    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.139313    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.139542    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.139676    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tffw9\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.139890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.139871    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241374    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241491    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241564    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241594    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241678    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241705    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tffw9\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.241772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.241740    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242099    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242170    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242301    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242342    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242541    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.242660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.242598    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.243156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.243124    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.246293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.245419    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.246293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.245601    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.249849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.249830    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.249953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.249931    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tffw9\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.401086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.399737    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:15.731046 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:16:15.731021    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc453299e_56e2_4cfe_ba3e_8c4bb86413c7.slice/crio-4c946f6a9a9e9457de416646b2915b37b9eec8cfef0b8ee6c3e952a6a4e08467 WatchSource:0}: Error finding container 4c946f6a9a9e9457de416646b2915b37b9eec8cfef0b8ee6c3e952a6a4e08467: Status 404 returned error can't find the container with id 4c946f6a9a9e9457de416646b2915b37b9eec8cfef0b8ee6c3e952a6a4e08467
Oct 30 21:16:15.731690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.731663    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:16:15.733881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.733829    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.733968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.733925    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.734004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.733983    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:15.959178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:15.959141    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:16.742539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:16.742503    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" event={"ID":"c453299e-56e2-4cfe-ba3e-8c4bb86413c7","Type":"ContainerStarted","Data":"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"}
Oct 30 21:16:16.744878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:16.742566    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" event={"ID":"c453299e-56e2-4cfe-ba3e-8c4bb86413c7","Type":"ContainerStarted","Data":"4c946f6a9a9e9457de416646b2915b37b9eec8cfef0b8ee6c3e952a6a4e08467"}
Oct 30 21:16:16.770343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:16.770259    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podStartSLOduration=1.770238889 podStartE2EDuration="1.770238889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:16.769186945 +0000 UTC m=+833.724874195" watchObservedRunningTime="2026-10-30 21:16:16.770238889 +0000 UTC m=+833.725926119"
Oct 30 21:16:16.876154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:16.876120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.415464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:17.407510    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:17.415464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:17.407590    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:17.415464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:17.407668    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:17.681036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:17.680951    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:17.966359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:17.966284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:18.412460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:18.406822    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:18.412460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:18.406894    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:19.115465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:19.107766    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:19.410516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:19.403828    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:19.410516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:19.403907    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:20.126463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:20.124979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.404512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:20.400788    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:20.404512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:20.400856    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:20.744629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:20.744525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:20.901271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:20.899971    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.337416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:21.337382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:21.402352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:21.402321    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:21.403049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:21.402643    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:21.946814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:21.946775    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:22.088792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:22.088733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:22.268678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:22.268591    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:22.401553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:22.401516    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 21:16:22.402101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:22.401608    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 21:16:22.512414 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:16:22.512376    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:16:23.050198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:23.050144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:23.406115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:23.406087    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:23.406551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:23.406473    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:23.407289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:23.407270    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:16:23.823990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:23.823906    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:24.527878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:24.527837    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:25.260196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:25.260165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:27.285460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.284547    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:27.461656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.461607    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Oct 30 21:16:27.631474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.619703    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": context deadline exceeded"
Oct 30 21:16:27.815765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.815553    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:27.897579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.884152    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:27.897579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.885343    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:27.897579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.885727    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:27.954519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:27.940761    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.186520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.180962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:28.214719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.214656    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.292922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.290697    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.292922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.291451    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:28.779710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.779667    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:16:28.787462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.780351    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.818022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.815826    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.818022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.816204    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.885828    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.886200    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.886485    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.886785    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.886845    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.946528    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.946978    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.959078    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:16:28.992461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.959462    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:28.993066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:28.992724    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:29.010537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.009413    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:29.765792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.765765    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:29.766372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.766354    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:29.767276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.767252    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:29.817413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.817384    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:29.817767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:29.817752    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:30.589779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.589739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:30.592150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.592129    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:16:30.613804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.613766    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:16:30.613973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.613886    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722059    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722097    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722273    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722322    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722419    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722516    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722535    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722550    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.722652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.722566    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn7cs\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.824419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824379    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.824419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824423    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.824973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824860    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.824973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824911    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824985    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825045    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825072    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825097    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825124    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn7cs\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825302    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825654    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825774    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.825994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.825883    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.826080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.824864    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.828962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.828925    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.828962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.828928    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.836535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.836503    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn7cs\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.837337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.837313    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:30.953645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:30.953616    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:31.247940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:31.247912    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:16:31.251502 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:16:31.251411    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99e4e11_1be4_4a3f_929d_ab29d50043f4.slice/crio-7f4ce023972d36c699c7dc43b124049c52da94385090ec117b232a58a4ef6a19 WatchSource:0}: Error finding container 7f4ce023972d36c699c7dc43b124049c52da94385090ec117b232a58a4ef6a19: Status 404 returned error can't find the container with id 7f4ce023972d36c699c7dc43b124049c52da94385090ec117b232a58a4ef6a19
Oct 30 21:16:31.264043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:31.264011    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.264124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:31.264082    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.264124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:31.264109    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:31.432780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:31.432726    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:32.137616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.137577    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" event={"ID":"f99e4e11-1be4-4a3f-929d-ab29d50043f4","Type":"ContainerStarted","Data":"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"}
Oct 30 21:16:32.138121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.137646    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" event={"ID":"f99e4e11-1be4-4a3f-929d-ab29d50043f4","Type":"ContainerStarted","Data":"7f4ce023972d36c699c7dc43b124049c52da94385090ec117b232a58a4ef6a19"}
Oct 30 21:16:32.162647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.162592    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podStartSLOduration=2.162574161 podStartE2EDuration="2.162574161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:32.160604125 +0000 UTC m=+849.116291354" watchObservedRunningTime="2026-10-30 21:16:32.162574161 +0000 UTC m=+849.118261392"
Oct 30 21:16:32.439288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.439212    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:32.954156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.954116    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:32.955142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.955117    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:32.955278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:32.955167    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:33.236707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:33.236631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.739110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:33.739068    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:33.954524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:33.954474    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:33.954702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:33.954554    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:34.289463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:34.286211    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:34.954809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:34.954777    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:34.955028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:34.954841    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:35.245011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:35.244930    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:35.954044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:35.954005    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:35.954552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:35.954079    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:36.002459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:36.001863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:36.833393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:36.833355    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:36.954913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:36.954886    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:36.955300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:36.954951    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:37.712249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:37.712208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:37.954269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:37.954236    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 21:16:37.954446 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:37.954286    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 21:16:38.537073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:38.537035    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:38.994869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:38.994838    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:38.995180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:38.995145    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:38.996382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:38.996353    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:16:39.442532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:39.442505    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:40.116096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:40.116058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:40.227610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:40.227572    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:40.823716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:40.823680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:41.046615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:41.046579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:41.426045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:41.425967    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:41.566456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:41.566377    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:42.120513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:42.120484    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:42.962297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:42.962264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:43.903489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:43.903458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:44.399330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.399230    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:44.596509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.596471    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:44.736634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.736534    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.769743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.769689    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.852031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.850950    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.852350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.852324    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:44.896795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:44.896757    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.013160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.013056    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.229207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.229170    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.262059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.262012    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:45.451462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.451314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:45.505466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.503590    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:45.505466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.503680    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.614658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.614615    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.615634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.615609    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:45.635474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.632587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:45.714517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.693395    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:16:45.737493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.737398    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.743638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.743610    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:45.749460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.744760    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:45.802660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.799548    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.823768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.817079    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:16:45.867033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.866987    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:45.873811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.867509    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.873811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.869062    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:45.873811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.869121    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:45.938469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.912533    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:16:45.938469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:45.913004    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:16:46.048386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.004991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:46.278770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.086518    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.284464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.168226    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmnp7\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.288461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.284973    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.292545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.292518    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.292787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.292767    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.303505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.293003    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.303709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.303685    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.303890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.303872    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.304055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.304039    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.304233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.304217    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.304532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.222563    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:46.326462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.189015    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.326682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.322490    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:46.406239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.406194    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.431668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.408689    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.431968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.431930    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.432155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.432127    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.432419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.432386    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.432571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.432555    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.432809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.432793    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.433021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.432986    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.433206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.433189    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.433644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.433621    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmnp7\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.459488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.434262    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.462679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.462651    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.478834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.478810    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.480188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.480151    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.649916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.642807    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:46.737694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.735392    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmnp7\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.743211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.743131    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.745059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.744007    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:46.853609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.853564    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 21:16:46.869557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.869511    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.873768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.873729    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.926521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.922504    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:46.967107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:46.967036    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\") pod \"e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") " pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:47.184601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.173263    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:47.247936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.226514    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.292131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.292072    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.456315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.454649    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Oct 30 21:16:47.645752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.644557    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.645752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.644910    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": context deadline exceeded"
Oct 30 21:16:47.652977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.652941    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:16:47.688805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.688778    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:47.721601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.720981    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:47.721601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.721094    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.932547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.926530    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.932547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.927520    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.932547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.928528    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:47.932547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.928976    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:48.000454 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:47.987190    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:48.342541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.127624    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.408462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.293660    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.482462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.467663    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:16:48.591463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.586633    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": context deadline exceeded"
Oct 30 21:16:48.643230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.641565    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:48.643230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.641639    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.679589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.660995    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.679589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.661494    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:48.938139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.932525    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.938139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.932726    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:16:48.938139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.932997    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.938139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.933126    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:48.965701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:48.960910    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.084276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.084233    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.198553 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:16:49.187095    6738 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc923af9_5b1f_4d22_8b7b_445d1d573027.slice/crio-4e09c66244f535d1477f4a08453fc74d8c51ae0f96dd2dbd6637c5d36a236a61\": RecentStats: unable to find data in memory cache]"
Oct 30 21:16:49.308896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.304528    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.342470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.323709    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:49.342470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.323830    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:49.342470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.323877    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:16:49.348651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.344527    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.348651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.346558    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:49.348959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.348929    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:49.356999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.356958    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.390906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.390853    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:16:49.488834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.477578    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.653238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.653198    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:49.668882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.668832    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.925468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.896290    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:49.949312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.949269    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.965369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.965309    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:49.966456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.966238    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.966862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.966831    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:16:49.995820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:49.967875    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.207449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.157163    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/health\": context deadline exceeded"
Oct 30 21:16:50.336522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.311778    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.392623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.392584    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.396864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.396668    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:50.397310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.397291    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:50.528633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.528530    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.676649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.669219    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.950567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.950519    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:50.979398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.968527    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.979398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.968878    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.979398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.978776    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:16:50.980823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:50.980252    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:51.071193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.071155    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" event={"ID":"fc923af9-5b1f-4d22-8b7b-445d1d573027","Type":"ContainerStarted","Data":"4e09c66244f535d1477f4a08453fc74d8c51ae0f96dd2dbd6637c5d36a236a61"}
Oct 30 21:16:51.102263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.102204    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podStartSLOduration=6.102188109 podStartE2EDuration="6.102188109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:16:51.100746199 +0000 UTC m=+868.056433429" watchObservedRunningTime="2026-10-30 21:16:51.102188109 +0000 UTC m=+868.057875336"
Oct 30 21:16:51.148076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.148043    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:16:51.230559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.220372    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:51.230559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.220455    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:51.349463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.347286    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:51.432564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.401322    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:51.483534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.475015    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:51.483534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.475122    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:16:51.591331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.589350    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": context deadline exceeded"
Oct 30 21:16:51.618262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.618234    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:16:51.622210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.622176    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:16:51.673161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.673116    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:16:51.704453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.704386    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:16:51.726895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.726862    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:16:51.814315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.814239    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:16:51.846570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:51.836195    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:16:52.074751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.074664    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:16:52.210466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.180678    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:52.210466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.180744    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:52.453285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.453250    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" event={"ID":"fc923af9-5b1f-4d22-8b7b-445d1d573027","Type":"ContainerStarted","Data":"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"}
Oct 30 21:16:52.693630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.690968    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:52.878648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:52.818232    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:52.928461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:16:52.919116    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:16:53.098492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.098449    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:53.291513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.287948    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:53.291513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.288005    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:53.335618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.329710    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:16:53.398463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.394503    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:16:53.588521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.586197    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-xcwwf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:16:53.588521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.586256    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-xcwwf" podUID="a04cf628-eece-472b-91c0-8b25c2407e57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:16:53.603464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.602259    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:16:53.658062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.654709    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:16:53.824472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:53.798892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:54.176499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:54.176459    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:54.176893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:54.176530    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:54.858464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:54.857841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.176569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:55.176531    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:55.176964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:55.176592    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:55.569347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:55.569256    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.732609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:55.728848    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:55.862092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:55.862063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:56.176903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:56.176815    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:56.177510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:56.177477    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:56.547787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:56.547699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:57.027078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:57.027042    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:57.160868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:57.160830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:57.176994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:57.176951    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:57.177406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:57.177038    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:58.088109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:58.088079    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.176724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:58.176694    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 21:16:58.176883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:58.176755    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 21:16:58.469704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:58.469663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:58.634460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:58.633743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.260798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.260760    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.737921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.737863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:16:59.827152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.827121    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:59.827927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.827903    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:59.828733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.828715    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:16:59.895171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:16:59.894086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:00.277702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.277659    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:17:00.283024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.283000    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.286225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.286195    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:17:00.293983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.293958    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-5bc8c60a-serving-cert\""
Oct 30 21:17:00.349223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.349186    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:00.364904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.364865    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.365050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.365028    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.365108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.365093    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.365173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.365158    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clrv8\" (UniqueName: \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.466208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.466173    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.466407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.466237    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.466407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.466303    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clrv8\" (UniqueName: \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.466407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.466382    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.467229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.467199    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.469205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.469182    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.469301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.469220    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.474883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.474858    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clrv8\" (UniqueName: \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\") pod \"maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.475660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.475638    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:00.594730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.594631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:00.601520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.601493    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:00.887227 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:17:00.887193    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c693d8_794f_4480_bd01_e93bcb20db10.slice/crio-d1bdf9a0278b55f9a25b30670ac4926e3fbdfbfdc4835bb970b786bb89cb5d76 WatchSource:0}: Error finding container d1bdf9a0278b55f9a25b30670ac4926e3fbdfbfdc4835bb970b786bb89cb5d76: Status 404 returned error can't find the container with id d1bdf9a0278b55f9a25b30670ac4926e3fbdfbfdc4835bb970b786bb89cb5d76
Oct 30 21:17:00.887610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.887280    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:17:00.891022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:00.891000    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:17:01.351144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:01.351055    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:01.697984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:01.697953    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" event={"ID":"75c693d8-794f-4480-bd01-e93bcb20db10","Type":"ContainerStarted","Data":"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"}
Oct 30 21:17:01.698143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:01.698018    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" event={"ID":"75c693d8-794f-4480-bd01-e93bcb20db10","Type":"ContainerStarted","Data":"d1bdf9a0278b55f9a25b30670ac4926e3fbdfbfdc4835bb970b786bb89cb5d76"}
Oct 30 21:17:01.698219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:01.698191    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:01.721724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:01.721668    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" podStartSLOduration=1.235664826 podStartE2EDuration="1.721649419s" totalImagesPullingTime="485.984593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:01.716894689 +0000 UTC m=+878.672581923" watchObservedRunningTime="2026-10-30 21:17:01.721649419 +0000 UTC m=+878.677336648"
Oct 30 21:17:02.181831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:02.181778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:04.601015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:04.600978    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:04.946297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:04.946252    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:04.980657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:04.980617    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:05.453470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.449626    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:17:05.462461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.453750    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:05.482463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.459591    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.516464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.490103    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6a24e1-serving-cert\""
Oct 30 21:17:05.516464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.510088    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:17:05.670463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.666517    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.670463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.666596    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9vh9\" (UniqueName: \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.670463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.666691    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.670463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.666725    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.771377    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9vh9\" (UniqueName: \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.771486    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.771520    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.771706    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:05.771863    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-serving-cert: secret "maas-api-e2e-api-a-6a24e1-serving-cert" not found
Oct 30 21:17:05.774550 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:05.771932    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls podName:492c4d94-bdbd-4798-816c-fd69d9e4799f nodeName:}" failed. No retries permitted until 2026-10-30 21:17:06.271912081 +0000 UTC m=+883.227599304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls") pod "maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f") : secret "maas-api-e2e-api-a-6a24e1-serving-cert" not found
Oct 30 21:17:05.851493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.848560    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:05.851493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.849397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:05.982463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.964378    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:05.982463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.980745    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:17:05.982733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.982478    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:06.132519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:05.983942    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:17:06.405517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.358591    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:06.405517 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:06.358893    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-serving-cert: secret "maas-api-e2e-api-a-6a24e1-serving-cert" not found
Oct 30 21:17:06.405517 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:06.358982    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls podName:492c4d94-bdbd-4798-816c-fd69d9e4799f nodeName:}" failed. No retries permitted until 2026-10-30 21:17:07.358961715 +0000 UTC m=+884.314648944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls") pod "maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f") : secret "maas-api-e2e-api-a-6a24e1-serving-cert" not found
Oct 30 21:17:06.432372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.415116    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:06.432722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.432694    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:17:06.432930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.432895    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:06.460491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.460097    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:06.460491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.460386    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:17:06.462787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.461969    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:06.464156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.464090    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9vh9\" (UniqueName: \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:06.504051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.480526    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:17:06.682514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.682245    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:17:06.694353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.684326    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:06.859210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.820310    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:17:06.945515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:06.927802    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:17:07.037462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.010597    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:07.038286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.038241    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:07.149469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.064383    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:17:07.149469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.088672    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:17:07.149469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.088732    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:07.437575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.437527    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Oct 30 21:17:07.439979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.439951    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:07.476203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.476174    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:07.512552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.512414    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:07.603072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.603040    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:17:07.633553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.631857    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:07.695871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.695760    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:07.697192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.697169    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:17:07.845464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.837792    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:17:07.907817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:07.907773    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:17:08.156649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.156601    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:08.284485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.279777    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:08.307696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.294527    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:17:08.568768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.568675    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:08.610084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.609516    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:08.655711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.650891    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:17:08.679306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.679055    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:17:08.871521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.843830    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:08.871521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.844396    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:17:08.872459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.872391    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-vgrr4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:17:08.872700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.872640    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-vgrr4" podUID="b791e110-a436-4eba-9674-7c0fded22180" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:09.028461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:08.998532    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:09.028461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.003241    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:17:09.098085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.098047    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" event={"ID":"492c4d94-bdbd-4798-816c-fd69d9e4799f","Type":"ContainerStarted","Data":"f503bd5d950862f74acc746d5da44ecdd7918a04cc1c97749356611dff2812be"}
Oct 30 21:17:09.163922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.162673    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:09.242363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.242336    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:17:09.311229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.309914    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:17:09.429625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.429520    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:17:09.459239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.456072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:09.680504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.679864    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:17:09.708575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:09.708544    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:17:10.376195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:10.373757    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:17:10.850858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:10.850826    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:11.246126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.246091    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" event={"ID":"492c4d94-bdbd-4798-816c-fd69d9e4799f","Type":"ContainerStarted","Data":"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"}
Oct 30 21:17:11.246339 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.246160    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:11.265350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.265300    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" podStartSLOduration=5.021426964 podStartE2EDuration="6.265287796s" totalImagesPullingTime="1.243860832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:11.262567108 +0000 UTC m=+888.218254335" watchObservedRunningTime="2026-10-30 21:17:11.265287796 +0000 UTC m=+888.220975023"
Oct 30 21:17:11.627557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.627529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:11.730139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.730105    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:11.835840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:11.835801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:12.562910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:12.562878    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:13.435672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:13.435636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:14.300470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:14.300406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:15.212111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:15.212074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:16.271698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:16.271661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:16.950979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:16.950945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:17.261501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:17.261393    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:17:17.746392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:17.746360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:17.923214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:17.923169    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:18.513715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:18.513681    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:19.215821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:19.215745    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:19.840733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:19.840699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.286242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:20.286196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.626136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:20.626106    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:20.890973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:20.890887    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:21.474899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:21.474816    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:21.592918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:21.592887    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:21.983598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:21.983554    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.046492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.046456    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:17:22.051673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.051649    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.062130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.062095    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:17:22.062742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.062711    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5bc8c60a\"/\"default-dockercfg-dv274\""
Oct 30 21:17:22.063250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.063194    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5bc8c60a\"/\"openshift-service-ca.crt\""
Oct 30 21:17:22.063372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.063289    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-5bc8c60a\"/\"kube-root-ca.crt\""
Oct 30 21:17:22.063520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.063405    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-5bc8c60a\"/\"e2e-lifecycle-model-5bc8c60a-kserve-self-signed-certs\""
Oct 30 21:17:22.125031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.124999    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.164685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.164640    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.164874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.164809    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.164944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.164891    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ld55m\" (UniqueName: \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.165002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.164956    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.165066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.165032    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.165163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.165139    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.241143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.241054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.270903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.270858    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.271320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.271298    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.271702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.271681    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ld55m\" (UniqueName: \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.271906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.271878    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.272100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.272085    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.272255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.272239    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.289086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.285602    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.289463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.289425    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.290078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.290006    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.290544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.290508    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.315100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.297389    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ld55m\" (UniqueName: \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.315100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.314462    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\") pod \"e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") " pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.383461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.379868    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:22.555691 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:22.555646    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:17:22.656822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.654759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:22.788037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:22.788014    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:17:22.792850 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:17:22.792483    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6541552c_a5ad_4e8f_b674_b8487c6e65c6.slice/crio-89ad19589e3327d500fc577a8667bb4cd4f2eec5537a7de35866759a986b6871 WatchSource:0}: Error finding container 89ad19589e3327d500fc577a8667bb4cd4f2eec5537a7de35866759a986b6871: Status 404 returned error can't find the container with id 89ad19589e3327d500fc577a8667bb4cd4f2eec5537a7de35866759a986b6871
Oct 30 21:17:23.373632 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:23.373593    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" event={"ID":"6541552c-a5ad-4e8f-b674-b8487c6e65c6","Type":"ContainerStarted","Data":"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"}
Oct 30 21:17:23.373791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:23.373646    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" event={"ID":"6541552c-a5ad-4e8f-b674-b8487c6e65c6","Type":"ContainerStarted","Data":"89ad19589e3327d500fc577a8667bb4cd4f2eec5537a7de35866759a986b6871"}
Oct 30 21:17:23.381427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:23.381375    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:23.386759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:23.386730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:23.398548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:23.398496    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" podStartSLOduration=1.398479862 podStartE2EDuration="1.398479862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:23.39625202 +0000 UTC m=+900.351939255" watchObservedRunningTime="2026-10-30 21:17:23.398479862 +0000 UTC m=+900.354167092"
Oct 30 21:17:24.529909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.529881    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"]
Oct 30 21:17:24.534793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.534765    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.552545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.552513    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 21:17:24.552688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.552560    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 21:17:24.552688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.552575    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 21:17:24.552897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.552880    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-z426l\""
Oct 30 21:17:24.567591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.567561    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"]
Oct 30 21:17:24.640337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640305    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640362    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea85e6a1-85a1-4a78-a913-705ad58be881-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640488    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640691    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640720    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56fnd\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-kube-api-access-56fnd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640843    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.640947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.640935    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742086    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742130    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea85e6a1-85a1-4a78-a913-705ad58be881-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742163    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742331    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742357    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56fnd\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-kube-api-access-56fnd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742426    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.742812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.742521    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.743105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.743084    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.745018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.744994    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea85e6a1-85a1-4a78-a913-705ad58be881-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.745673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.745615    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.745673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.745630    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.745673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.745638    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ea85e6a1-85a1-4a78-a913-705ad58be881-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.776917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.776876    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56fnd\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-kube-api-access-56fnd\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.785276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.785182    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea85e6a1-85a1-4a78-a913-705ad58be881-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-f7hgs\" (UID: \"ea85e6a1-85a1-4a78-a913-705ad58be881\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:24.796320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.796291    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:24.852949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:24.852923    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:25.151365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:25.151334    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"]
Oct 30 21:17:25.153526 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:17:25.153490    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea85e6a1_85a1_4a78_a913_705ad58be881.slice/crio-6149234e814c2934bdaf14a89a4f4940ddc882fc6e3e37c0bd2f13e7ab50f70b WatchSource:0}: Error finding container 6149234e814c2934bdaf14a89a4f4940ddc882fc6e3e37c0bd2f13e7ab50f70b: Status 404 returned error can't find the container with id 6149234e814c2934bdaf14a89a4f4940ddc882fc6e3e37c0bd2f13e7ab50f70b
Oct 30 21:17:25.393016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:25.392972    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs" event={"ID":"ea85e6a1-85a1-4a78-a913-705ad58be881","Type":"ContainerStarted","Data":"6149234e814c2934bdaf14a89a4f4940ddc882fc6e3e37c0bd2f13e7ab50f70b"}
Oct 30 21:17:25.757569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:25.757532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:26.201640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:26.201600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:26.324011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:26.323982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.070457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:27.054126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.401584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:27.401546    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.525706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:27.525670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:27.967646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:27.967614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:28.682555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:28.682516    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:28.682877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:28.682589    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:17:29.066482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.066344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:29.446798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.446491    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs" event={"ID":"ea85e6a1-85a1-4a78-a913-705ad58be881","Type":"ContainerStarted","Data":"aef56fe24f0f16ac18e246aaafe82efb79277bff0d7840d33e9d2665fc8f9889"}
Oct 30 21:17:29.447016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.446830    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:29.448858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.448832    6738 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-f7hgs container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:17:29.449025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.448888    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs" podUID="ea85e6a1-85a1-4a78-a913-705ad58be881" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:17:29.473381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.472483    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs" podStartSLOduration=1.947256069 podStartE2EDuration="5.472466851s" totalImagesPullingTime="3.525210782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:17:29.468921995 +0000 UTC m=+906.424609225" watchObservedRunningTime="2026-10-30 21:17:29.472466851 +0000 UTC m=+906.428154080"
Oct 30 21:17:29.911494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:29.911458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:30.453803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:30.453768    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-f7hgs"
Oct 30 21:17:30.672057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:30.665327    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:31.546661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:31.546629    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.346561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:32.346528    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:32.381539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:32.381507    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:32.400905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:32.400874    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:32.482881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:32.482853    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:17:33.049710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:33.049667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:33.835419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:33.835376    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:33.945688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:33.945654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.059794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:34.059763    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.634777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:34.634693    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:34.853160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:34.853132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:35.131116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:35.131082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:35.592580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:35.592419    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:36.086086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:36.086053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:37.000977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:37.000941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:37.682902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:37.682858    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:38.524825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:38.524785    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:38.942656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:38.942606    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:39.064651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:39.064614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:39.345752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:39.345660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:40.156696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:40.156653    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:40.292063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:40.292029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:41.118324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:41.118291    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:41.874101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:41.874072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:42.652159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:42.652126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:43.444620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:43.444567    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:44.445043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:44.445014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:45.237720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:45.237689    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.024629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:46.024586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.329693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:46.329616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.803362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:46.803327    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:46.919686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:46.919633    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:47.875504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:47.875465    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:48.899663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:48.899629    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:49.732186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:49.732147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:50.353842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:50.353798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:51.027900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:51.027870    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:51.785240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:51.785207    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:52.450455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:52.450401    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:52.533677 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:17:52.533638    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:17:53.145861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:53.145833    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:53.887775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:53.887736    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:54.533288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:54.533255    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:55.232387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:55.232350    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:56.054654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:56.054552    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:56.774017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:56.773966    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:57.541012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:57.540974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:58.575035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:58.574997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:58.685421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:58.685367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:58.966465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:58.962979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:17:59.521128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:17:59.521086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:00.232984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:00.232936    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:00.923481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:00.923427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:01.830256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:01.830223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:02.622317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:02.622289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:03.337864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:03.337830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:04.076505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:04.076464    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:04.857606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:04.857579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:05.533732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:05.533701    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:06.302175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:06.302139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:06.776916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:06.776880    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:18:06.777209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:06.777181    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerName="maas-api" containerID="cri-o://8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a" gracePeriod=30
Oct 30 21:18:06.849313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:06.849271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.128237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.128044    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.236221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.236183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.286317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.286285    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:18:07.330247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.330214    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\" (UniqueName: \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\") pod \"492c4d94-bdbd-4798-816c-fd69d9e4799f\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") "
Oct 30 21:18:07.331308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.331285    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\") pod \"492c4d94-bdbd-4798-816c-fd69d9e4799f\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") "
Oct 30 21:18:07.332016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.332000    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\") pod \"492c4d94-bdbd-4798-816c-fd69d9e4799f\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") "
Oct 30 21:18:07.332537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.331848    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca" pod "492c4d94-bdbd-4798-816c-fd69d9e4799f" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:18:07.332795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.332773    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") pod \"492c4d94-bdbd-4798-816c-fd69d9e4799f\" (UID: \"492c4d94-bdbd-4798-816c-fd69d9e4799f\") "
Oct 30 21:18:07.336822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.335290    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/492c4d94-bdbd-4798-816c-fd69d9e4799f-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:18:07.342969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.342932    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9" pod "492c4d94-bdbd-4798-816c-fd69d9e4799f" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f"). InnerVolumeSpecName "kube-api-access-x9vh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:18:07.347075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.344938    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls" pod "492c4d94-bdbd-4798-816c-fd69d9e4799f" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:07.347075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.345954    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs" pod "492c4d94-bdbd-4798-816c-fd69d9e4799f" (UID: "492c4d94-bdbd-4798-816c-fd69d9e4799f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:18:07.436941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.436881    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9vh9\" (UniqueName: \"kubernetes.io/projected/492c4d94-bdbd-4798-816c-fd69d9e4799f-kube-api-access-x9vh9\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:18:07.436941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.436907    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:18:07.436941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.436917    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/492c4d94-bdbd-4798-816c-fd69d9e4799f-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:18:07.493254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.493222    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.610458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.607283    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:07.777930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.777854    6738 generic.go:356] "Generic (PLEG): container finished" podID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerID="8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a" exitCode=0
Oct 30 21:18:07.777930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.777908    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"
Oct 30 21:18:07.778238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.777936    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" event={"ID":"492c4d94-bdbd-4798-816c-fd69d9e4799f","Type":"ContainerDied","Data":"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"}
Oct 30 21:18:07.778238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.777991    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" event={"ID":"492c4d94-bdbd-4798-816c-fd69d9e4799f","Type":"ContainerDied","Data":"f503bd5d950862f74acc746d5da44ecdd7918a04cc1c97749356611dff2812be"}
Oct 30 21:18:07.778238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.778012    6738 scope.go:122] "RemoveContainer" containerID="8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"
Oct 30 21:18:07.791308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.791290    6738 scope.go:122] "RemoveContainer" containerID="8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"
Oct 30 21:18:07.791632 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:18:07.791609    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a\": container with ID starting with 8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a not found: ID does not exist" containerID="8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"
Oct 30 21:18:07.791714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.791641    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a"} err="failed to get container status \"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a\": rpc error: code = NotFound desc = could not find container \"8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a\": container with ID starting with 8198f41b20021eb87d296496fc1daf54577cadbd3ccd2b86da528f4fefb1c31a not found: ID does not exist"
Oct 30 21:18:07.803761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.803739    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:18:07.807010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:07.806987    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq"]
Oct 30 21:18:08.156386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:08.156347    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:08.815523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:08.815487    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:09.165711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:09.165682    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" path="/var/lib/kubelet/pods/492c4d94-bdbd-4798-816c-fd69d9e4799f/volumes"
Oct 30 21:18:09.518059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:09.517981    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:10.261826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:10.261789    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6a24e1-7fff647474-7d2rq" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.71:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:10.278128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:10.278102    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:10.926710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:10.926676    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:11.257047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:11.256973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:11.363127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:11.363099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:11.807011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:11.806974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:12.443054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:12.443026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:13.130420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:13.130389    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:13.825108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:13.825074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.047621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.047591    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.257398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.257365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.520773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.520690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.661461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.657695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.783078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.782998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:14.890795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:14.890764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:15.640671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:15.640639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:16.328179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:16.328147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:17.048109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:17.048083    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:17.870956 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:17.870198    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:18.133570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:18.133495    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:18.646702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:18.646675    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:19.587376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:19.587341    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:20.248823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:20.248791    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:20.916394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:20.916353    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.029254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:21.029217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:21.648249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:21.648210    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:22.528975 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:18:22.527979    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:18:22.533673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:22.533640    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:23.237121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:23.237090    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:23.955557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:23.955524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:24.585128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:24.585099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.149275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:25.149241    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.255938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:25.255907    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:25.536726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:25.536623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:26.264697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:26.264669    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:27.031632 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:27.031581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:27.747274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:27.747242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.415212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:28.415142    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.601452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:28.601400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.769052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:28.768977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:28.881532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:28.881498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.330823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:29.330795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:29.647566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:29.647536    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:30.379709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:30.379658    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:31.046409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:31.046365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:31.813067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:31.813032    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:32.832832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:32.832797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:33.724555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:33.724522    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:34.558017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:34.557986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.090383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:35.090351    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.268045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:35.268003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.392463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:35.387566    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.572082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:35.572050    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:35.936523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:35.936489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.448277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:36.448239    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.585261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:36.585229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.721526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:36.721278    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:36.873101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:36.873063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.177683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:37.177654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.295465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:37.294838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:37.516520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:37.515196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:38.390877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:38.390779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:39.223166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:39.223121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:40.445960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:40.445916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:40.874054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:40.874001    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:41.009025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:41.008931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:41.712459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:41.711519    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-zqthr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:18:41.712459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:41.711579    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:18:41.854713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:41.854674    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:42.330155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:42.330114    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:42.475381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:42.475342    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:42.804459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:42.804392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:42.932496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:42.928979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:43.236785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:43.236742    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:43.460246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:43.460217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:43.836042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:43.836008    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:44.534953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:44.534565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:45.357034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:45.356996    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:46.040604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:46.040573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:46.742845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:46.742812    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.548659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:47.548622    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.670505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:47.670472    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:47.770881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:47.770847    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:48.569649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:48.569616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:49.348999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:49.348968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:50.018148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:50.018078    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:50.701570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:50.701542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:51.320663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:51.320619    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:52.025094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:52.025058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:52.509983 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:18:52.509934    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:18:52.648918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:52.648889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:53.350409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:53.350375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:54.150974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:54.150946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:54.830657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:54.830624    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:55.543958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:55.543923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.222577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:56.222543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:56.863082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:56.863051    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:57.546849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:57.546801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:58.415558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:58.415527    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:18:59.425015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:18:59.424987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:00.373761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:00.373723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:00.940476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:00.935835    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:01.269067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:01.268937    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:01.607319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:01.607183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:02.166224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:02.166182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:02.319460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:02.317348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:02.576168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:02.576076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:03.070461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:03.068169    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:03.779555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:03.778928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:03.915733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:03.915698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:04.158032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:04.157998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:04.542980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:04.542755    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:05.977403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:05.977359    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.167214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:06.167183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.303426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:06.299486    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:06.777010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:06.776965    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:19:06.785789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:06.785752    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" podUID="75c693d8-794f-4480-bd01-e93bcb20db10" containerName="maas-api" containerID="cri-o://a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f" gracePeriod=30
Oct 30 21:19:06.828452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:06.828295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.314261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.314227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.440225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.440192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.563506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.562749    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:19:07.564587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.564520    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.565138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.564890    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" event={"ID":"75c693d8-794f-4480-bd01-e93bcb20db10","Type":"ContainerDied","Data":"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"}
Oct 30 21:19:07.565313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.565298    6738 scope.go:122] "RemoveContainer" containerID="a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"
Oct 30 21:19:07.568685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.564564    6738 generic.go:356] "Generic (PLEG): container finished" podID="75c693d8-794f-4480-bd01-e93bcb20db10" containerID="a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f" exitCode=0
Oct 30 21:19:07.568685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.568583    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w" event={"ID":"75c693d8-794f-4480-bd01-e93bcb20db10","Type":"ContainerDied","Data":"d1bdf9a0278b55f9a25b30670ac4926e3fbdfbfdc4835bb970b786bb89cb5d76"}
Oct 30 21:19:07.579150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.579131    6738 scope.go:122] "RemoveContainer" containerID="a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"
Oct 30 21:19:07.579744 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:19:07.579705    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f\": container with ID starting with a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f not found: ID does not exist" containerID="a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"
Oct 30 21:19:07.579947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.579920    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f"} err="failed to get container status \"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f\": rpc error: code = NotFound desc = could not find container \"a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f\": container with ID starting with a5beff584542ee60d0deea135479c260dd8644523e9505f4430f85ccad623e9f not found: ID does not exist"
Oct 30 21:19:07.601915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.601885    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\" (UniqueName: \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\") pod \"75c693d8-794f-4480-bd01-e93bcb20db10\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") "
Oct 30 21:19:07.602173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.602152    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\") pod \"75c693d8-794f-4480-bd01-e93bcb20db10\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") "
Oct 30 21:19:07.602644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.602626    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\") pod \"75c693d8-794f-4480-bd01-e93bcb20db10\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") "
Oct 30 21:19:07.602789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.602777    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\") pod \"75c693d8-794f-4480-bd01-e93bcb20db10\" (UID: \"75c693d8-794f-4480-bd01-e93bcb20db10\") "
Oct 30 21:19:07.605947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.605925    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca" pod "75c693d8-794f-4480-bd01-e93bcb20db10" (UID: "75c693d8-794f-4480-bd01-e93bcb20db10"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:19:07.631112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.631074    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8" pod "75c693d8-794f-4480-bd01-e93bcb20db10" (UID: "75c693d8-794f-4480-bd01-e93bcb20db10"). InnerVolumeSpecName "kube-api-access-clrv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:19:07.631539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.631368    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls" pod "75c693d8-794f-4480-bd01-e93bcb20db10" (UID: "75c693d8-794f-4480-bd01-e93bcb20db10"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:07.631780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.631743    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs" pod "75c693d8-794f-4480-bd01-e93bcb20db10" (UID: "75c693d8-794f-4480-bd01-e93bcb20db10"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:19:07.705071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.704988    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clrv8\" (UniqueName: \"kubernetes.io/projected/75c693d8-794f-4480-bd01-e93bcb20db10-kube-api-access-clrv8\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:19:07.705071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.705019    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:19:07.705071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.705033    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75c693d8-794f-4480-bd01-e93bcb20db10-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:19:07.705071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.705046    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75c693d8-794f-4480-bd01-e93bcb20db10-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:19:07.726495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.726264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:07.858640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:07.858428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:08.586016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:08.585990    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"
Oct 30 21:19:08.625747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:08.624206    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:19:08.630019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:08.629325    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-5bc8c60a-6cd9986bdb-w2r8w"]
Oct 30 21:19:08.732420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:08.732385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:09.177467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:09.177406    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75c693d8-794f-4480-bd01-e93bcb20db10" path="/var/lib/kubelet/pods/75c693d8-794f-4480-bd01-e93bcb20db10/volumes"
Oct 30 21:19:09.568715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:09.564387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.247049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:10.247015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.727413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:10.727374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:10.852710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:10.852679    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.057496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:11.057397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.185460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:11.183104    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.592130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:11.592096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:11.884071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:11.884038    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:12.958232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:12.958202    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:13.222842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:13.222762    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:13.861898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:13.861843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:14.165146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:14.165067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:14.722493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:14.722425    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:15.546376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:15.546319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.218925 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:16.218894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.424973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:16.424947    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.679305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:16.679265    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:16.795812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:16.792550    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:17.228628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:17.228596    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.131565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:18.131529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.241950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:18.241911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:18.896563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:18.896525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.010817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:19.010780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.279488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:19.279366    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.403366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:19.403334    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:19.945875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:19.945847    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:20.311121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:20.311036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:20.465340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:20.465307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.156063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:21.156031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.553532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:21.553420    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:21.667718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:21.667683    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:22.405637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:22.405604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:22.520200 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:19:22.520161    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:19:23.339687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:23.339646    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:24.055856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:24.055805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:24.376761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:24.376730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:24.881504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:24.881234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:25.033895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:25.029320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:25.407897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:25.407853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:26.180541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:26.177540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:26.602356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:26.602234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:26.762474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:26.747355    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:27.576460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:27.574102    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:27.993461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:27.991269    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:19:28.643452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:28.643355    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:28.935639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:28.935561    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:29.073635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:29.073606    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:29.760997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:29.760960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.118158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:30.118133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.347758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:30.347703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.645563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:30.645532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.773858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:30.773831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:30.917411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:30.917329    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:31.651558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:31.651524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:32.537775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:32.537739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.219650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:33.219613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:33.906189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:33.906156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:34.554835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:34.554802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:35.243621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:35.243580    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:35.930573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:35.930544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:36.991672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:36.991630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.112341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:37.112306    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.232684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:37.232642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.425817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:37.425746    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:37.538747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:37.538715    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.118661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:38.118633    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:38.829854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:38.829819    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:39.726122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:39.726072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:40.557939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:40.557905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:41.296388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:41.296360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.123711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:42.123675    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:42.941019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:42.940989    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:43.637396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:43.637366    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:44.534949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:44.534913    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:44.736394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:44.736333    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:44.882837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:44.882802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.535457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:45.534778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.672293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:45.672234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:45.851066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:45.850980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:46.672508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:46.672476    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:47.410865 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:47.410823    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:48.413065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:48.413030    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:48.925706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:48.925672    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.053751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:49.053708    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.184901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:49.184801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.306181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:49.306136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:49.546978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:49.546892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.147940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:50.147911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:50.846070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:50.846029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:51.637778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:51.637743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:52.338367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:52.338340    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:52.509630 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:19:52.509594    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:19:53.018833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:53.018803    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:53.701271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:53.701221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.195691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:54.195650    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.323606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:54.323573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.818696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:54.818665    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:54.992891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:54.992860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.108370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:55.108279    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.516866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:55.516827    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.628795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:55.628753    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.822536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:55.822422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:55.984242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:55.984203    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:56.645802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:56.645774    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:57.550215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:57.545650    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:58.226515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:58.226340    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:58.355153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:58.355120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:58.927922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:58.927877    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:19:59.742060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:19:59.742029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:00.426111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:00.426076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.007836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:01.007806    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.210895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:01.210849    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.365177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:01.365151    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.623973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:01.623879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:01.802144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:01.802108    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.524589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:02.522892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.659997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:02.659970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.841924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:02.841807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:02.965935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:02.965896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:03.727538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:03.727498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:04.436077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:04.436041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:05.237469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:05.237417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.042460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:06.042392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.734873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:06.734837    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:20:06.762010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:06.761980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:06.887494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:06.887407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:07.450369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:07.450276    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:07.551599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:07.551550    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:07.986484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:07.983878    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.121186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:08.121132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:08.338832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:08.338727    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:09.248619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.248583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:09.915602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.915574    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:20:09.919181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.918422    6738 cpu_manager.go:414] "containerMap: removing container" podUID="75c693d8-794f-4480-bd01-e93bcb20db10" containerName="maas-api"
Oct 30 21:20:09.919181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.918464    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75c693d8-794f-4480-bd01-e93bcb20db10" containerName="maas-api"
Oct 30 21:20:09.919181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.918485    6738 cpu_manager.go:414] "containerMap: removing container" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerName="maas-api"
Oct 30 21:20:09.919181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.918493    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerName="maas-api"
Oct 30 21:20:09.920290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.920267    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="75c693d8-794f-4480-bd01-e93bcb20db10" containerName="maas-api"
Oct 30 21:20:09.920417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.920295    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="492c4d94-bdbd-4798-816c-fd69d9e4799f" containerName="maas-api"
Oct 30 21:20:09.925650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.925629    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:09.926848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.926819    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:20:09.933083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:09.933062    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-3ac6bf-openshift-default-dockercfg-vznzc\""
Oct 30 21:20:10.033397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033363    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033416    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033510    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033547    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033569    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033727    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fk8mj\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033765    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033842    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.033912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.033900    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.124902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.124860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:10.135094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135061    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135156    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135202    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135354    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135447    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135481    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135601    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fk8mj\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135651    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135666    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135708    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.135813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.135742    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.136040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.136013    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.136212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.136088    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.136620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.136597    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.138090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.138071    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.138622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.138606    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.142839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.142815    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.143749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.143723    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fk8mj\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\") pod \"e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") " pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.262898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.262818    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:10.564586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.564554    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:20:10.566297 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:20:10.566261    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07bfe52_18db_4a07_a2d6_5ac71d41e8ae.slice/crio-783bb6399816c4dc7e4328becd11638fae45f9d3438a64c79afe924af654033f WatchSource:0}: Error finding container 783bb6399816c4dc7e4328becd11638fae45f9d3438a64c79afe924af654033f: Status 404 returned error can't find the container with id 783bb6399816c4dc7e4328becd11638fae45f9d3438a64c79afe924af654033f
Oct 30 21:20:10.569562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.569518    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.569690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.569604    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.569690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.569647    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:20:10.867724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.867670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:10.993008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:10.992977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:11.338516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:11.338041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:11.531461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:11.521972    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" event={"ID":"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae","Type":"ContainerStarted","Data":"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"}
Oct 30 21:20:11.531461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:11.522063    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" event={"ID":"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae","Type":"ContainerStarted","Data":"783bb6399816c4dc7e4328becd11638fae45f9d3438a64c79afe924af654033f"}
Oct 30 21:20:11.911462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:11.900931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.108464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.103210    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.231526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.230938    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:12.263601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.263571    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:12.264660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.264640    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:20:12.264732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.264688    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 21:20:12.420619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:12.420113    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:13.146733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.146698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:13.263588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.263554    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:20:13.263907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.263610    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 21:20:13.319476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.319269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:13.626792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.626701    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:13.839458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:13.837122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:14.264041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:14.263999    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:20:14.264226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:14.264054    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 21:20:14.305460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:14.300976    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:14.861426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:14.858196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:15.015464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:15.014451    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:15.274814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:15.274719    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:20:15.275098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:15.275064    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 21:20:15.924359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:15.923804    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.264223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:16.264126    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 21:20:16.264381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:16.264223    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 21:20:16.299995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:16.299914    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:16.433675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:16.433574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:17.191261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.191225    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:17.269059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.269019    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:17.269326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.269309    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:17.270203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.270185    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:20:17.292572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.292511    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podStartSLOduration=8.292493975 podStartE2EDuration="8.292493975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:11.582730304 +0000 UTC m=+1068.538417534" watchObservedRunningTime="2026-10-30 21:20:17.292493975 +0000 UTC m=+1074.248181204"
Oct 30 21:20:17.341614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.341588    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:17.842906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:17.842866    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:18.169727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:18.169693    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:18.544335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:18.544248    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.068382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:19.068348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.168630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:19.168595    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.340327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:19.340248    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:19.977569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:19.977532    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:20:20.015793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.015758    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.024015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.023184    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-43f6ab-serving-cert\""
Oct 30 21:20:20.074557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.066466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:20.182778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.182743    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:20:20.197760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.197717    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.197954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.197876    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.198031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.197977    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfnp2\" (UniqueName: \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.198091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.198067    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.218355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.218321    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:20.299706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.299624    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.299855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.299830    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.299974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.299960    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.300011 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:20:20.299987    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-serving-cert: secret "maas-api-e2e-ipp-b-43f6ab-serving-cert" not found
Oct 30 21:20:20.300048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.300007    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfnp2\" (UniqueName: \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.300088 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:20:20.300070    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls podName:b8cd1623-f72c-4c94-880a-0249905f2aa2 nodeName:}" failed. No retries permitted until 2026-10-30 21:20:20.800046603 +0000 UTC m=+1077.755733883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls") pod "maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" (UID: "b8cd1623-f72c-4c94-880a-0249905f2aa2") : secret "maas-api-e2e-ipp-b-43f6ab-serving-cert" not found
Oct 30 21:20:20.300295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.300277    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.302985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.302959    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.310501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.310479    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfnp2\" (UniqueName: \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.460207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.460158    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:20.811574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.811530    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:20.841844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:20.841806    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:21.045390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:21.045099    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:21.235977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:21.235849    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:21.366647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:21.366615    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:20:21.371956 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:20:21.371925    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8cd1623_f72c_4c94_880a_0249905f2aa2.slice/crio-5dd56482c817171cb0e1bf8a175ef3114b4d248271ca2a595eb1475b09705b8b WatchSource:0}: Error finding container 5dd56482c817171cb0e1bf8a175ef3114b4d248271ca2a595eb1475b09705b8b: Status 404 returned error can't find the container with id 5dd56482c817171cb0e1bf8a175ef3114b4d248271ca2a595eb1475b09705b8b
Oct 30 21:20:21.659148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:21.659044    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" event={"ID":"b8cd1623-f72c-4c94-880a-0249905f2aa2","Type":"ContainerStarted","Data":"5dd56482c817171cb0e1bf8a175ef3114b4d248271ca2a595eb1475b09705b8b"}
Oct 30 21:20:21.757493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:21.757416    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" podUID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" containerName="main" containerID="cri-o://68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81" gracePeriod=45
Oct 30 21:20:22.041238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.041218    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:20:22.059576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059552    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.059950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059625    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\" (UniqueName: \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.059950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059676    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.059950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059808    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\" (UniqueName: \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.059950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059845    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.059950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.059928    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\") pod \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\" (UID: \"6541552c-a5ad-4e8f-b674-b8487c6e65c6\") "
Oct 30 21:20:22.060306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.060266    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:22.060551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.060457    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:22.060725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.060687    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.060725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.060716    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.060938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.060817    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:22.062342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.062315    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "kube-api-access-ld55m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:20:22.062539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.062521    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:20:22.062931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.062909    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs" pod "6541552c-a5ad-4e8f-b674-b8487c6e65c6" (UID: "6541552c-a5ad-4e8f-b674-b8487c6e65c6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:20:22.117647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.117612    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:22.161768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.161740    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.161768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.161766    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ld55m\" (UniqueName: \"kubernetes.io/projected/6541552c-a5ad-4e8f-b674-b8487c6e65c6-kube-api-access-ld55m\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.162008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.161776    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.162008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.161792    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6541552c-a5ad-4e8f-b674-b8487c6e65c6-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:20:22.509787 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:20:22.509745    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:20:22.667194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.667102    6738 generic.go:356] "Generic (PLEG): container finished" podID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" containerID="68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81" exitCode=0
Oct 30 21:20:22.667194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.667154    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"
Oct 30 21:20:22.667452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.667189    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" event={"ID":"6541552c-a5ad-4e8f-b674-b8487c6e65c6","Type":"ContainerDied","Data":"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"}
Oct 30 21:20:22.667452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.667234    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4" event={"ID":"6541552c-a5ad-4e8f-b674-b8487c6e65c6","Type":"ContainerDied","Data":"89ad19589e3327d500fc577a8667bb4cd4f2eec5537a7de35866759a986b6871"}
Oct 30 21:20:22.667452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.667258    6738 scope.go:122] "RemoveContainer" containerID="68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"
Oct 30 21:20:22.669608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.669581    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" event={"ID":"b8cd1623-f72c-4c94-880a-0249905f2aa2","Type":"ContainerStarted","Data":"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"}
Oct 30 21:20:22.669739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.669722    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:22.681978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.681959    6738 scope.go:122] "RemoveContainer" containerID="68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"
Oct 30 21:20:22.682267 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:20:22.682249    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81\": container with ID starting with 68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81 not found: ID does not exist" containerID="68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"
Oct 30 21:20:22.682333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.682279    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81"} err="failed to get container status \"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81\": rpc error: code = NotFound desc = could not find container \"68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81\": container with ID starting with 68915bb2fe13f1183bb087142ee2cd1b7623dbfe7692332e011f231b3d64dd81 not found: ID does not exist"
Oct 30 21:20:22.693196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.693157    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" podStartSLOduration=3.223663457 podStartE2EDuration="3.693145187s" totalImagesPullingTime="469.48173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:22.692213847 +0000 UTC m=+1079.647901078" watchObservedRunningTime="2026-10-30 21:20:22.693145187 +0000 UTC m=+1079.648832415"
Oct 30 21:20:22.713158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.713135    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:20:22.720167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:22.720138    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-5bc8c60a/e2e-lifecycle-model-5bc8c60a-kserve-5cd695dcfc-qjqv4"]
Oct 30 21:20:23.165732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:23.165705    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" path="/var/lib/kubelet/pods/6541552c-a5ad-4e8f-b674-b8487c6e65c6/volumes"
Oct 30 21:20:23.184717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:23.184695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:23.312076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:23.312041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:23.471490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:23.471391    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.169882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:24.169848    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.354329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:24.354296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.504513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:24.503905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.844671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:24.844595    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:24.953183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:24.953139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:25.063653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:25.063617    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:25.757926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:25.757898    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:26.450008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:26.449972    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:27.196122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:27.196087    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:28.030738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:28.030698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:28.684539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:28.684513    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:20:28.840029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:28.839999    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:29.539570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:29.539537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:30.538373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:30.538337    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:31.543345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:31.543310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.126336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:32.126303    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.246223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:32.246189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.507612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:32.507521    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:32.691106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:32.691068    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:33.619427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:33.619384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:33.781835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:33.781792    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:33.999964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:33.999873    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:34.123523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:34.123489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:34.557692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:34.557609    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:34.875607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:34.875568    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:35.014573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:35.014543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:35.178598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:35.177402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:35.524687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:35.524602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:36.466238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:36.466132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:37.219066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:37.219024    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:37.345582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:37.345549    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:38.347509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:38.347468    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:38.479126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:38.479091    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:38.645379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:38.645338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:39.269556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:39.269502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:39.738354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:39.738299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:39.857976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:39.857944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:40.331760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:40.331730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:41.048664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:41.048618    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:41.976485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:41.976445    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.519121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:42.519050    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:42.669706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:42.669636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:43.196226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:43.196180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:43.604695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:43.604600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.030817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:44.030784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.554735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:44.554698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.678722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:44.678685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:44.831960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:44.831865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:45.550421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.550375    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:20:45.551726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.551700    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" containerName="main"
Oct 30 21:20:45.551726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.551722    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" containerName="main"
Oct 30 21:20:45.551950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.551839    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6541552c-a5ad-4e8f-b674-b8487c6e65c6" containerName="main"
Oct 30 21:20:45.556814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.556792    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.568231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.568172    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2a55fb-serving-cert\""
Oct 30 21:20:45.579221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.577626    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:20:45.680245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.680215    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcqkn\" (UniqueName: \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.680654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.680260    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.680654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.680282    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.680654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.680422    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.708026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.707995    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:45.781524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.781488    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.781795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.781774    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcqkn\" (UniqueName: \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.781878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.781835    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.781878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.781866    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.782859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.782831    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.784742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.784717    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.784920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.784889    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.792558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.792529    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcqkn\" (UniqueName: \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\") pod \"maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:45.842007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.841890    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:45.891649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:45.891619    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:46.292344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.291925    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:20:46.629047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.629022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:46.887320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.887223    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" event={"ID":"73714aac-7469-4213-a13d-53291c20d0fb","Type":"ContainerStarted","Data":"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"}
Oct 30 21:20:46.887320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.887280    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" event={"ID":"73714aac-7469-4213-a13d-53291c20d0fb","Type":"ContainerStarted","Data":"0a9b5595ebcd39210f1ef3d631c8eb15a35cc7338f84d1bd4697f482a1cb21b5"}
Oct 30 21:20:46.887913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.887449    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:46.905902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:46.905857    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" podStartSLOduration=1.507274286 podStartE2EDuration="1.905844762s" totalImagesPullingTime="398.570476ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:20:46.903405495 +0000 UTC m=+1103.859092723" watchObservedRunningTime="2026-10-30 21:20:46.905844762 +0000 UTC m=+1103.861531990"
Oct 30 21:20:47.541978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:47.541941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:47.648087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:47.648055    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:48.450588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:48.450550    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:49.309265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:49.309226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.237174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:50.237139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.624027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:50.623997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.741749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:50.741710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:50.984784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:50.984685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.105133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:51.104063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.773776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:51.773728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:51.967307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:51.967273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:52.374951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:52.374921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:52.501460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:52.501308    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:52.655926 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:20:52.655846    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:20:52.943578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:52.940943    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:20:53.333174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:53.333096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:54.142504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:54.142417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:54.905302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:54.905270    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:55.661172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:55.661139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:55.884270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:55.884230    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.020525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:56.016459    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.263835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:56.263773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:56.902982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:56.902939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:57.853844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:57.853805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:57.990741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:57.990706    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.139462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:58.126511    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.294761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:58.293488    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.428158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:58.428071    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:58.605080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:58.605043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:59.153890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:59.153863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:20:59.863978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:20:59.863944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:00.650448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:00.650411    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:01.444494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:01.444419    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:02.421369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:02.421335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:03.710405 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:03.710373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:04.355259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:04.355232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.143626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.143583    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:21:05.149570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.149549    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.162556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.162532    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-dbc08f-serving-cert\""
Oct 30 21:21:05.176831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.176810    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:21:05.233866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.233836    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.265236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.265191    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.265485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.265264    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b77x\" (UniqueName: \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.265485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.265302    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.265485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.265326    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.346071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.346043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.366816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.366783    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.366997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.366832    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b77x\" (UniqueName: \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.366997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.366866    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.366997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.366888    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.367171 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:21:05.367037    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-serving-cert: secret "maas-api-e2e-api-a-dbc08f-serving-cert" not found
Oct 30 21:21:05.367171 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:21:05.367147    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls podName:c40bce38-3e85-418a-8309-a7247b7a70c5 nodeName:}" failed. No retries permitted until 2026-10-30 21:21:05.867126283 +0000 UTC m=+1122.822813505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls") pod "maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" (UID: "c40bce38-3e85-418a-8309-a7247b7a70c5") : secret "maas-api-e2e-api-a-dbc08f-serving-cert" not found
Oct 30 21:21:05.367771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.367746    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.369907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.369884    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.379429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.379397    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b77x\" (UniqueName: \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.541414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.541319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:05.887463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.886845    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:05.900302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:05.899483    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") pod \"maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:06.087426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:06.087399    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:06.127509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:06.127420    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:06.393944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:06.393916    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:21:06.396196 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:21:06.396168    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40bce38_3e85_418a_8309_a7247b7a70c5.slice/crio-1c8dd65656d6b9c616c0b0cd13a6993d388502ac2a795ad13ff779b671e4113a WatchSource:0}: Error finding container 1c8dd65656d6b9c616c0b0cd13a6993d388502ac2a795ad13ff779b671e4113a: Status 404 returned error can't find the container with id 1c8dd65656d6b9c616c0b0cd13a6993d388502ac2a795ad13ff779b671e4113a
Oct 30 21:21:06.930012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:06.929981    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:07.322578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.322540    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" event={"ID":"c40bce38-3e85-418a-8309-a7247b7a70c5","Type":"ContainerStarted","Data":"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"}
Oct 30 21:21:07.322773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.322610    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:07.322773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.322638    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" event={"ID":"c40bce38-3e85-418a-8309-a7247b7a70c5","Type":"ContainerStarted","Data":"1c8dd65656d6b9c616c0b0cd13a6993d388502ac2a795ad13ff779b671e4113a"}
Oct 30 21:21:07.341822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.341777    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" podStartSLOduration=1.9189484129999999 podStartE2EDuration="2.341762027s" totalImagesPullingTime="422.813614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:21:07.340380621 +0000 UTC m=+1124.296067839" watchObservedRunningTime="2026-10-30 21:21:07.341762027 +0000 UTC m=+1124.297449254"
Oct 30 21:21:07.490990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.490953    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:07.636227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:07.636197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.130764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:08.130735    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:08.828618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:08.828580    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:09.854989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:09.854957    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:10.548743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:10.548706    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:11.542750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:11.542716    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:12.341745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:12.341713    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:13.140639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:13.140611    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:13.336314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:13.336290    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:21:14.030665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:14.030630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:14.762337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:14.762306    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:15.662617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:15.662587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:16.503788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:16.503747    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:17.275263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:17.275234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.178251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:18.178216    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.299149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:18.299123    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:18.898976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:18.898944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.048515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:19.046965    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.179222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:19.179141    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.308250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:19.308219    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.424092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:19.424059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:19.572471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:19.572344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:20.080584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:20.080551    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:20.644955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:20.644924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.052505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:21.052417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.567907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:21.567879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:21.711281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:21.711245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:22.184992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:22.184961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:22.512580 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:21:22.512481    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:21:22.581919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:22.581884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:22.862829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:22.862800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:23.542136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:23.542103    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.347667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:24.347631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.550568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:24.550532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:24.985171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:24.985137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:25.276520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:25.276263    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:25.571759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:25.571670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:25.702378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:25.702341    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:25.927262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:25.927228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:26.241993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:26.241892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:26.891899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:26.891864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.019311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.019263    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.139461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.138637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.486363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.486330    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.631032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.631001    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:27.775008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.774918    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:21:27.775253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.775167    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" containerID="cri-o://eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047" gracePeriod=30
Oct 30 21:21:27.837918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:27.837887    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.411303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:28.411273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.745853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:28.745757    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:28.868407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:28.868373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:29.774379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:29.774349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:29.979101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:29.979045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:30.111756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:30.111720    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:30.850966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:30.850934    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:31.665963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:31.665935    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:32.271969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:32.271928    6738 patch_prober.go:29] interesting pod/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:21:32.272477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:32.272014    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:21:32.557271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:32.557193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.075329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.075275    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:21:33.148318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148278    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148367    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148403    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148429    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148477    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148582    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148680    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148715    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:21:33.148760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148755    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.148965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148768    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:21:33.148965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.148784    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\") pod \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\" (UID: \"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae\") "
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149181    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149325    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149339    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149707    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149728    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149743    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149757    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.149863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.149773    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.151714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.151665    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:21:33.151714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.151674    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:21:33.151714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.151687    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:21:33.151946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.151766    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj" pod "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" (UID: "e07bfe52-18db-4a07-a2d6-5ac71d41e8ae"). InnerVolumeSpecName "kube-api-access-fk8mj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:21:33.251229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.251171    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.251428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.251409    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.251572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.251474    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.251572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.251491    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fk8mj\" (UniqueName: \"kubernetes.io/projected/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae-kube-api-access-fk8mj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:21:33.336365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.336260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:33.553907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.553876    6738 generic.go:356] "Generic (PLEG): container finished" podID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerID="eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047" exitCode=0
Oct 30 21:21:33.554101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.553954    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"
Oct 30 21:21:33.554101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.553956    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" event={"ID":"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae","Type":"ContainerDied","Data":"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"}
Oct 30 21:21:33.554101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.554017    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk" event={"ID":"e07bfe52-18db-4a07-a2d6-5ac71d41e8ae","Type":"ContainerDied","Data":"783bb6399816c4dc7e4328becd11638fae45f9d3438a64c79afe924af654033f"}
Oct 30 21:21:33.554101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.554040    6738 scope.go:122] "RemoveContainer" containerID="eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"
Oct 30 21:21:33.567977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.567957    6738 scope.go:122] "RemoveContainer" containerID="eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"
Oct 30 21:21:33.568423 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:21:33.568402    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047\": container with ID starting with eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047 not found: ID does not exist" containerID="eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"
Oct 30 21:21:33.568542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.568447    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047"} err="failed to get container status \"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047\": rpc error: code = NotFound desc = could not find container \"eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047\": container with ID starting with eb976e84bf1f96f1ca65afd3167fa32857050efc493027cdfcca9bf791af4047 not found: ID does not exist"
Oct 30 21:21:33.573647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.573624    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:21:33.577224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:33.577202    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-3ac6bf-openshift-default-c786786b9-wtdvk"]
Oct 30 21:21:34.142191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:34.142151    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:35.073720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:35.073670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:35.172978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:35.172942    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" path="/var/lib/kubelet/pods/e07bfe52-18db-4a07-a2d6-5ac71d41e8ae/volumes"
Oct 30 21:21:35.726133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:35.726090    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:35.844247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:35.844215    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:36.087183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:36.087096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:36.528161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:36.528131    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:37.194096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:37.194062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:37.869175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:37.869147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:38.561448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.561405    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:21:38.562479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.562424    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy"
Oct 30 21:21:38.562479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.562468    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy"
Oct 30 21:21:38.562669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.562655    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e07bfe52-18db-4a07-a2d6-5ac71d41e8ae" containerName="istio-proxy"
Oct 30 21:21:38.568355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.568329    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.573524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.573502    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-09eb50-serving-cert\""
Oct 30 21:21:38.576755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.576718    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:21:38.637295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.637254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:38.639836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.639813    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.639943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.639851    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72h2t\" (UniqueName: \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.639943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.639922    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.640056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.640036    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.741647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.741604    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72h2t\" (UniqueName: \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.741851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.741693    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.741851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.741824    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.742102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.742066    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.743370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.743345    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.745498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.745131    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.749167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.749139    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.752933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.752832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:38.753609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.753579    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72h2t\" (UniqueName: \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\") pod \"maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:38.887050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:38.887020    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:39.274160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:39.274124    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:21:39.553835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:39.553750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:39.622326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:39.622294    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" event={"ID":"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f","Type":"ContainerStarted","Data":"7531f77fd3ae2b5d07c116c4ba403e0305b9deab5eee1d654fdeb6192cf678fe"}
Oct 30 21:21:40.537879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:40.537822    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:40.632991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:40.632943    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" event={"ID":"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f","Type":"ContainerStarted","Data":"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"}
Oct 30 21:21:40.633506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:40.633031    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:40.653487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:40.650911    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" podStartSLOduration=2.180418643 podStartE2EDuration="2.650890742s" totalImagesPullingTime="470.472099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:21:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:21:40.650087221 +0000 UTC m=+1157.605774472" watchObservedRunningTime="2026-10-30 21:21:40.650890742 +0000 UTC m=+1157.606577971"
Oct 30 21:21:41.456559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:41.456463    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:42.152757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:42.152727    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:42.927996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:42.927954    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:43.622164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:43.622130    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.323998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.323965    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.443483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.439274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.557133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.557098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.719144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.719107    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.879085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.879052    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:44.997888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:44.997787    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:45.382734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:45.382680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:45.765340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:45.765235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:46.647665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:46.647635    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:21:46.760653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:46.760608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:47.554174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:47.554132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.229727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:48.229674    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.697714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:48.697676    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:48.940870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:48.940833    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:49.401962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:49.401930    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:50.041754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:50.038680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:50.156061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:50.156026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:50.780096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:50.780061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.097776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:51.097690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:51.931062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:51.931014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:52.195576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:52.195475    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:52.321292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:52.321259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:52.532718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:52.532602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:52.538693 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:21:52.538661    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:21:53.079442 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:53.079400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:53.534207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:53.534175    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:53.670920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:53.670886    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.037323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:54.037271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.144720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:54.144681    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.491493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:54.484728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:54.915888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:54.915847    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:55.926165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:55.926130    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:56.557952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:56.557896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:57.333710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:57.333668    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:58.032638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:58.032602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:58.788372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:58.788336    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.650546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:59.650488    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.774802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:59.774772    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:21:59.904854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:21:59.904765    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:00.736579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:00.736535    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:01.139651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:01.139613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:01.262096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:01.262057    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:01.462111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:01.462014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:01.609963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:01.609926    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.240033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:02.239986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.744888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:02.744861    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:02.866758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:02.866723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:03.268493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:03.268457    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.020881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.020845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.654397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.654307    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:22:04.662819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.661092    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.672796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.672765    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-6563b6-serving-cert\""
Oct 30 21:22:04.672939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.672920    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:22:04.731699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.731643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.754268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.754235    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp69d\" (UniqueName: \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.754421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.754274    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.754512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.754456    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.754572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.754521    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.832771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.832731    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:04.856126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.856088    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.856315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.856141    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.856315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.856306    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp69d\" (UniqueName: \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.856448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.856324    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.856510 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:04.856473    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-serving-cert: secret "maas-api-e2e-praxis-a-6563b6-serving-cert" not found
Oct 30 21:22:04.856569 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:04.856551    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls podName:869444b0-ff99-4315-afdc-69e7ca5c658d nodeName:}" failed. No retries permitted until 2026-10-30 21:22:05.35653207 +0000 UTC m=+1182.312219291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls") pod "maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" (UID: "869444b0-ff99-4315-afdc-69e7ca5c658d") : secret "maas-api-e2e-praxis-a-6563b6-serving-cert" not found
Oct 30 21:22:04.856811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.856791    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.858996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.858969    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:04.865278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:04.865253    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp69d\" (UniqueName: \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:05.048100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.048028    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:05.370510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.366062    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:05.391596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.385366    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") pod \"maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:05.585809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.585777    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:05.818367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.818329    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:05.885933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.885905    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:22:05.887195 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:22:05.887173    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869444b0_ff99_4315_afdc_69e7ca5c658d.slice/crio-0a25207ef594116ad28cd8f4cf06ad0f0866070ca02676a1f735fe2dc098885e WatchSource:0}: Error finding container 0a25207ef594116ad28cd8f4cf06ad0f0866070ca02676a1f735fe2dc098885e: Status 404 returned error can't find the container with id 0a25207ef594116ad28cd8f4cf06ad0f0866070ca02676a1f735fe2dc098885e
Oct 30 21:22:05.888620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.888601    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:22:05.955376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:05.955317    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" event={"ID":"869444b0-ff99-4315-afdc-69e7ca5c658d","Type":"ContainerStarted","Data":"0a25207ef594116ad28cd8f4cf06ad0f0866070ca02676a1f735fe2dc098885e"}
Oct 30 21:22:06.709744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:06.709710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:06.966769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:06.966638    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" event={"ID":"869444b0-ff99-4315-afdc-69e7ca5c658d","Type":"ContainerStarted","Data":"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"}
Oct 30 21:22:06.966769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:06.966718    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:06.986260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:06.986217    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" podStartSLOduration=2.539907324 podStartE2EDuration="2.986204362s" totalImagesPullingTime="446.297038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:06.983487346 +0000 UTC m=+1183.939174578" watchObservedRunningTime="2026-10-30 21:22:06.986204362 +0000 UTC m=+1183.941891635"
Oct 30 21:22:07.474517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:07.474475    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:08.418187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:08.418148    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:09.122114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:09.122084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:09.791662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:09.791632    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.511535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:10.511503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.762920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:10.762843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:10.899598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:10.899560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:11.612859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:11.612827    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.480028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:12.479996    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:12.979745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:12.979719    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:22:13.484479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:13.484404    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:14.597527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:14.597478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:15.249242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:15.249212    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:15.954002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:15.953938    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:16.952302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:16.952268    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.272462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:18.272134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.467227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:18.467196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:18.746642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:18.746615    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:19.221160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:19.221126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.009761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:20.009728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.144191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:20.144152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:20.494101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:20.494054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.083218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:21.083174    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.206870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:21.206840    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:21.963081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:21.963046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:22.510391 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:22.510348    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:22:22.773064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:22.772973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:23.645003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:23.644970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:24.642635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:24.642604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:24.904828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:24.904729    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:25.095605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:25.095567    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:25.262878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:25.262798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.256586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:26.256556    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.521518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:26.507933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.655501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:26.655476    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:26.920649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:26.920613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:27.445101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:27.445059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:27.766042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:27.765959    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:28.732145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:28.732115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:29.524617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:29.524581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.068473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.068416    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:22:30.069013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.068684    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" podUID="c40bce38-3e85-418a-8309-a7247b7a70c5" containerName="maas-api" containerID="cri-o://6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867" gracePeriod=30
Oct 30 21:22:30.143833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.143768    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.446707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.446316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.606856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.605550    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:22:30.688790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.688762    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\" (UniqueName: \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\") pod \"c40bce38-3e85-418a-8309-a7247b7a70c5\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") "
Oct 30 21:22:30.689035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.688938    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\") pod \"c40bce38-3e85-418a-8309-a7247b7a70c5\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") "
Oct 30 21:22:30.689035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.689009    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") pod \"c40bce38-3e85-418a-8309-a7247b7a70c5\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") "
Oct 30 21:22:30.689198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.689047    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\") pod \"c40bce38-3e85-418a-8309-a7247b7a70c5\" (UID: \"c40bce38-3e85-418a-8309-a7247b7a70c5\") "
Oct 30 21:22:30.690228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.690097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:30.690887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.690859    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca" pod "c40bce38-3e85-418a-8309-a7247b7a70c5" (UID: "c40bce38-3e85-418a-8309-a7247b7a70c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:30.692336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.692305    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs" pod "c40bce38-3e85-418a-8309-a7247b7a70c5" (UID: "c40bce38-3e85-418a-8309-a7247b7a70c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:22:30.692801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.692778    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x" pod "c40bce38-3e85-418a-8309-a7247b7a70c5" (UID: "c40bce38-3e85-418a-8309-a7247b7a70c5"). InnerVolumeSpecName "kube-api-access-2b77x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:30.692801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.692779    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls" pod "c40bce38-3e85-418a-8309-a7247b7a70c5" (UID: "c40bce38-3e85-418a-8309-a7247b7a70c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:22:30.791808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.791715    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b77x\" (UniqueName: \"kubernetes.io/projected/c40bce38-3e85-418a-8309-a7247b7a70c5-kube-api-access-2b77x\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:30.791808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.791754    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c40bce38-3e85-418a-8309-a7247b7a70c5-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:30.791808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.791776    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:30.791808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.791791    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c40bce38-3e85-418a-8309-a7247b7a70c5-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:30.829133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:30.829099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:31.177807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.177779    6738 generic.go:356] "Generic (PLEG): container finished" podID="c40bce38-3e85-418a-8309-a7247b7a70c5" containerID="6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867" exitCode=0
Oct 30 21:22:31.178148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.177828    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"
Oct 30 21:22:31.178148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.177857    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" event={"ID":"c40bce38-3e85-418a-8309-a7247b7a70c5","Type":"ContainerDied","Data":"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"}
Oct 30 21:22:31.178148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.177897    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j" event={"ID":"c40bce38-3e85-418a-8309-a7247b7a70c5","Type":"ContainerDied","Data":"1c8dd65656d6b9c616c0b0cd13a6993d388502ac2a795ad13ff779b671e4113a"}
Oct 30 21:22:31.178148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.177918    6738 scope.go:122] "RemoveContainer" containerID="6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"
Oct 30 21:22:31.183037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.183014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:31.188529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.188511    6738 scope.go:122] "RemoveContainer" containerID="6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"
Oct 30 21:22:31.188809 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:31.188781    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867\": container with ID starting with 6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867 not found: ID does not exist" containerID="6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"
Oct 30 21:22:31.188927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.188819    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867"} err="failed to get container status \"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867\": rpc error: code = NotFound desc = could not find container \"6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867\": container with ID starting with 6312ff4d1c156e5bc7a466bf280180597c75f6555ce4408f9477ce9e5df61867 not found: ID does not exist"
Oct 30 21:22:31.201031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.200988    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:22:31.204037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:31.204016    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-dbc08f-f4877c644-xlg8j"]
Oct 30 21:22:32.020651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.020621    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.064859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.064827    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:22:32.065576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.065555    6738 cpu_manager.go:414] "containerMap: removing container" podUID="c40bce38-3e85-418a-8309-a7247b7a70c5" containerName="maas-api"
Oct 30 21:22:32.065576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.065576    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40bce38-3e85-418a-8309-a7247b7a70c5" containerName="maas-api"
Oct 30 21:22:32.065802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.065788    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40bce38-3e85-418a-8309-a7247b7a70c5" containerName="maas-api"
Oct 30 21:22:32.070725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.070705    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.076312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.076291    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-80e936-dockercfg-lc66f\""
Oct 30 21:22:32.076489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.076357    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-80e936\""
Oct 30 21:22:32.078562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.078542    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:22:32.129487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.119984    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:22:32.129487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.125283    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.129487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.129168    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:22:32.160381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.160338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.242151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.241318    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxb78\" (UniqueName: \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.242151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.241401    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.242151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.241507    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg8wm\" (UniqueName: \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.242151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.241708    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.343697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.343575    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.344304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.344276    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg8wm\" (UniqueName: \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.344653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.344633    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.344937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.344917    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxb78\" (UniqueName: \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.346524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.346404    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.347597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.346540    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.363160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.360617    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg8wm\" (UniqueName: \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\") pod \"payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.363346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.363322    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxb78\" (UniqueName: \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\") pod \"payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.385605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.385534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:32.389918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.389892    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:32.446817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.446786    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:32.688690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.688665    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:22:32.689020 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:22:32.688995    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb31a1b55_93f7_4316_b95a_58b16decb7e9.slice/crio-7a4a0d1a0cc234d30d7d5f0d4ca9bd40cf5f64d1790b6bb8587604f9d005d7b8 WatchSource:0}: Error finding container 7a4a0d1a0cc234d30d7d5f0d4ca9bd40cf5f64d1790b6bb8587604f9d005d7b8: Status 404 returned error can't find the container with id 7a4a0d1a0cc234d30d7d5f0d4ca9bd40cf5f64d1790b6bb8587604f9d005d7b8
Oct 30 21:22:32.786644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:32.786606    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:22:32.793424 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:22:32.793388    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2a166f_5fa2_4696_9ed2_7758b2f3b44e.slice/crio-d2116abecf8aaa2af700eae893bbbe56f9ff13fe1b7c4885daff886af425db17 WatchSource:0}: Error finding container d2116abecf8aaa2af700eae893bbbe56f9ff13fe1b7c4885daff886af425db17: Status 404 returned error can't find the container with id d2116abecf8aaa2af700eae893bbbe56f9ff13fe1b7c4885daff886af425db17
Oct 30 21:22:33.147172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:33.147144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:33.174506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:33.174295    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c40bce38-3e85-418a-8309-a7247b7a70c5" path="/var/lib/kubelet/pods/c40bce38-3e85-418a-8309-a7247b7a70c5/volumes"
Oct 30 21:22:33.206429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:33.206394    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" event={"ID":"b31a1b55-93f7-4316-b95a-58b16decb7e9","Type":"ContainerStarted","Data":"7a4a0d1a0cc234d30d7d5f0d4ca9bd40cf5f64d1790b6bb8587604f9d005d7b8"}
Oct 30 21:22:33.208062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:33.208031    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" event={"ID":"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e","Type":"ContainerStarted","Data":"d2116abecf8aaa2af700eae893bbbe56f9ff13fe1b7c4885daff886af425db17"}
Oct 30 21:22:34.141917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:34.141822    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:34.880507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:34.877177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:35.872396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:35.871407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:36.725242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:36.725204    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:37.248146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.248100    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" event={"ID":"b31a1b55-93f7-4316-b95a-58b16decb7e9","Type":"ContainerStarted","Data":"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"}
Oct 30 21:22:37.248667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.248408    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:37.250812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.250764    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" event={"ID":"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e","Type":"ContainerStarted","Data":"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"}
Oct 30 21:22:37.250931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.250856    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:37.269058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.269008    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" podStartSLOduration=1.888180346 podStartE2EDuration="5.268990761s" totalImagesPullingTime="3.380810415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:37.265671024 +0000 UTC m=+1214.221358253" watchObservedRunningTime="2026-10-30 21:22:37.268990761 +0000 UTC m=+1214.224677989"
Oct 30 21:22:37.446733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:37.446685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.123919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:38.123882    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.296358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:38.296325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.497701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:38.497604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:38.922194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:38.922135    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:39.600575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:39.600513    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:40.471989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.471922    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" podStartSLOduration=5.104964227 podStartE2EDuration="8.47190051s" totalImagesPullingTime="3.366936283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:37.283791533 +0000 UTC m=+1214.239478758" watchObservedRunningTime="2026-10-30 21:22:40.47190051 +0000 UTC m=+1217.427587739"
Oct 30 21:22:40.472728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.472676    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:22:40.474313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.474280    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" podUID="73714aac-7469-4213-a13d-53291c20d0fb" containerName="maas-api" containerID="cri-o://7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86" gracePeriod=30
Oct 30 21:22:40.543141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.543108    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:40.665897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.665855    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:40.854480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:40.854398    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.008509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.008148    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.077072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.077048    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:22:41.164957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.164930    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\") pod \"73714aac-7469-4213-a13d-53291c20d0fb\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") "
Oct 30 21:22:41.165226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.165199    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\") pod \"73714aac-7469-4213-a13d-53291c20d0fb\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") "
Oct 30 21:22:41.165325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.165307    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\" (UniqueName: \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\") pod \"73714aac-7469-4213-a13d-53291c20d0fb\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") "
Oct 30 21:22:41.165601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.165345    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\") pod \"73714aac-7469-4213-a13d-53291c20d0fb\" (UID: \"73714aac-7469-4213-a13d-53291c20d0fb\") "
Oct 30 21:22:41.165706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.165613    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca" pod "73714aac-7469-4213-a13d-53291c20d0fb" (UID: "73714aac-7469-4213-a13d-53291c20d0fb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:22:41.166146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.166129    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/73714aac-7469-4213-a13d-53291c20d0fb-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:41.168294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.168268    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls" pod "73714aac-7469-4213-a13d-53291c20d0fb" (UID: "73714aac-7469-4213-a13d-53291c20d0fb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:22:41.168397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.168272    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn" pod "73714aac-7469-4213-a13d-53291c20d0fb" (UID: "73714aac-7469-4213-a13d-53291c20d0fb"). InnerVolumeSpecName "kube-api-access-zcqkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:22:41.168496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.168470    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs" pod "73714aac-7469-4213-a13d-53291c20d0fb" (UID: "73714aac-7469-4213-a13d-53291c20d0fb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:22:41.192745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.192718    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:41.268211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.268128    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:41.268211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.268163    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcqkn\" (UniqueName: \"kubernetes.io/projected/73714aac-7469-4213-a13d-53291c20d0fb-kube-api-access-zcqkn\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:41.268211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.268177    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/73714aac-7469-4213-a13d-53291c20d0fb-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:22:41.281651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.281617    6738 generic.go:356] "Generic (PLEG): container finished" podID="73714aac-7469-4213-a13d-53291c20d0fb" containerID="7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86" exitCode=0
Oct 30 21:22:41.281838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.281797    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" event={"ID":"73714aac-7469-4213-a13d-53291c20d0fb","Type":"ContainerDied","Data":"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"}
Oct 30 21:22:41.281903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.281848    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m" event={"ID":"73714aac-7469-4213-a13d-53291c20d0fb","Type":"ContainerDied","Data":"0a9b5595ebcd39210f1ef3d631c8eb15a35cc7338f84d1bd4697f482a1cb21b5"}
Oct 30 21:22:41.281903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.281876    6738 scope.go:122] "RemoveContainer" containerID="7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"
Oct 30 21:22:41.282045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.282032    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"
Oct 30 21:22:41.299887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.299859    6738 scope.go:122] "RemoveContainer" containerID="7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"
Oct 30 21:22:41.301118 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:41.301074    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86\": container with ID starting with 7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86 not found: ID does not exist" containerID="7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"
Oct 30 21:22:41.301306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.301280    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86"} err="failed to get container status \"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86\": rpc error: code = NotFound desc = could not find container \"7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86\": container with ID starting with 7f75e51747439d3c8e9244759d3d8e195d2c5bab2304b0b4b1173ba81448ef86 not found: ID does not exist"
Oct 30 21:22:41.309288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.308988    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:22:41.309502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.309480    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2a55fb-6ccf5d4789-98q4m"]
Oct 30 21:22:41.591194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:41.591074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:42.023911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.023875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:42.656473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.656401    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:22:42.657521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.657497    6738 cpu_manager.go:414] "containerMap: removing container" podUID="73714aac-7469-4213-a13d-53291c20d0fb" containerName="maas-api"
Oct 30 21:22:42.657521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.657522    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73714aac-7469-4213-a13d-53291c20d0fb" containerName="maas-api"
Oct 30 21:22:42.657751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.657735    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="73714aac-7469-4213-a13d-53291c20d0fb" containerName="maas-api"
Oct 30 21:22:42.663610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.663576    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.673828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.673786    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:22:42.729309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.729273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:42.754718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.754686    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-80e936-serving-cert\""
Oct 30 21:22:42.796061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.796022    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.796277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.796124    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.796277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.796214    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-775qb\" (UniqueName: \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.796277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.796249    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.842165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.842124    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:42.897683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.897653    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-775qb\" (UniqueName: \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.897875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.897694    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.897955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.897937    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.898042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.898018    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.898103 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:42.898055    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-serving-cert: secret "maas-api-e2e-praxis-b-80e936-serving-cert" not found
Oct 30 21:22:42.898153 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:42.898123    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls podName:984f011c-adf9-4e70-b46c-bcbf79ad0b0a nodeName:}" failed. No retries permitted until 2026-10-30 21:22:43.398101486 +0000 UTC m=+1220.353788706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls") pod "maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" (UID: "984f011c-adf9-4e70-b46c-bcbf79ad0b0a") : secret "maas-api-e2e-praxis-b-80e936-serving-cert" not found
Oct 30 21:22:42.898794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.898758    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.901129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.901102    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:42.909418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:42.909348    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-775qb\" (UniqueName: \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:43.029190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.029156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:43.164791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.164708    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73714aac-7469-4213-a13d-53291c20d0fb" path="/var/lib/kubelet/pods/73714aac-7469-4213-a13d-53291c20d0fb/volumes"
Oct 30 21:22:43.410239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.410207    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:43.418896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.418249    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") pod \"maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:43.584968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.584933    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:43.650853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.650789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:43.905404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:43.893000    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:22:43.905404 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:22:43.897360    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod984f011c_adf9_4e70_b46c_bcbf79ad0b0a.slice/crio-657a1ea54409e1734a780f0e99d9f9f6f5dfdceff9a9aada1ecb71bf7827b040 WatchSource:0}: Error finding container 657a1ea54409e1734a780f0e99d9f9f6f5dfdceff9a9aada1ecb71bf7827b040: Status 404 returned error can't find the container with id 657a1ea54409e1734a780f0e99d9f9f6f5dfdceff9a9aada1ecb71bf7827b040
Oct 30 21:22:44.330279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:44.330252    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" event={"ID":"984f011c-adf9-4e70-b46c-bcbf79ad0b0a","Type":"ContainerStarted","Data":"657a1ea54409e1734a780f0e99d9f9f6f5dfdceff9a9aada1ecb71bf7827b040"}
Oct 30 21:22:44.445083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:44.445047    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:45.340576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:45.340538    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" event={"ID":"984f011c-adf9-4e70-b46c-bcbf79ad0b0a","Type":"ContainerStarted","Data":"6883ebf39c4e5183923a76db7d63680acaa9b49af0d94e46353980b20c771330"}
Oct 30 21:22:45.341071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:45.340636    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:45.365387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:45.365325    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" podStartSLOduration=2.98941066 podStartE2EDuration="3.365308358s" totalImagesPullingTime="375.897698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:22:45.362760374 +0000 UTC m=+1222.318447603" watchObservedRunningTime="2026-10-30 21:22:45.365308358 +0000 UTC m=+1222.320995627"
Oct 30 21:22:45.440563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:45.440532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.137776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:46.137743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:46.888512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:46.888473    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:47.846870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:47.846808    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.263649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.263604    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:22:48.264162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.263680    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:22:48.345301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.345260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.484029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.483989    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.658166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.658024    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:48.804462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:48.803358    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.169231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:49.169152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.299542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:49.299511    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:49.901295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:49.901238    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:50.737413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:50.737373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:51.356683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:51.356648    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:22:51.782139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:51.782080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:52.518275 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:22:52.518239    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:22:52.825100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:52.825006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:53.629881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:53.629830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:54.334728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:54.334699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:55.297314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:55.297274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:55.427978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:55.427947    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:55.830732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:55.830701    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:56.267910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:56.267879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.306204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:57.306172    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:57.928590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:57.928505    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.055515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:58.055482    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.180990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:58.180872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.338492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:58.333304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.535021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:58.534893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:58.992503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:58.992469    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:22:59.394009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:22:59.393978    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:00.190871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:00.190831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:00.313982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:00.313948    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:01.163607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:01.163576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:02.285844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:02.285808    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:02.881065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:02.881030    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:23:02.881304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:02.881276    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" podUID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" containerName="maas-api" containerID="cri-o://b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b" gracePeriod=30
Oct 30 21:23:02.958285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:02.958254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.081129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.081049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.347464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.280699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.530916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.530826    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.536564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.536543    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:23:03.610649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.610592    6738 generic.go:356] "Generic (PLEG): container finished" podID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" containerID="b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b" exitCode=0
Oct 30 21:23:03.611058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.611034    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" event={"ID":"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f","Type":"ContainerDied","Data":"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"}
Oct 30 21:23:03.611274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.611257    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5" event={"ID":"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f","Type":"ContainerDied","Data":"7531f77fd3ae2b5d07c116c4ba403e0305b9deab5eee1d654fdeb6192cf678fe"}
Oct 30 21:23:03.611427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.611413    6738 scope.go:122] "RemoveContainer" containerID="b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"
Oct 30 21:23:03.611788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.611773    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"
Oct 30 21:23:03.628360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.628302    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\") pod \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") "
Oct 30 21:23:03.628853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.628831    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\") pod \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") "
Oct 30 21:23:03.628960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.628893    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\" (UniqueName: \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\") pod \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") "
Oct 30 21:23:03.629058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.629045    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\") pod \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\" (UID: \"0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f\") "
Oct 30 21:23:03.629280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.629249    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca" pod "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" (UID: "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:03.629707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.629682    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:03.633347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.633290    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls" pod "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" (UID: "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:23:03.634931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.634907    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t" pod "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" (UID: "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f"). InnerVolumeSpecName "kube-api-access-72h2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:03.643460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.641534    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs" pod "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" (UID: "0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:23:03.656958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.656831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:03.659086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.658859    6738 scope.go:122] "RemoveContainer" containerID="b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"
Oct 30 21:23:03.660463 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:23:03.659267    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b\": container with ID starting with b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b not found: ID does not exist" containerID="b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"
Oct 30 21:23:03.660463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.659304    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b"} err="failed to get container status \"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b\": rpc error: code = NotFound desc = could not find container \"b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b\": container with ID starting with b47ff7921869bf414de34eca1a5fda31f11c6983f99849cfc0d98448d929430b not found: ID does not exist"
Oct 30 21:23:03.747578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.733631    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:03.747916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.747891    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72h2t\" (UniqueName: \"kubernetes.io/projected/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-kube-api-access-72h2t\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:03.748035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.748021    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:03.939885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.939852    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:23:03.947862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:03.947838    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-09eb50-54774fbb4-wpzw5"]
Oct 30 21:23:04.025878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:04.025797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:04.160059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:04.160029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:04.655460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:04.654704    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:05.166000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:05.165965    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" path="/var/lib/kubelet/pods/0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f/volumes"
Oct 30 21:23:05.426292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:05.426214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:06.131759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:06.131703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:06.912328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:06.912288    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:07.728911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:07.728877    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:08.500261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:08.500221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.142885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:09.142839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.256690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:09.256636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.364322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:09.364281    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:09.814893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:09.814853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.316993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:10.316962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.437829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:10.437788    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.693461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:10.691157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.817459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:10.817402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:10.931662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:10.931630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.341635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:11.341596    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.463198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:11.463155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.732261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:11.732228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:11.856494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:11.855929    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:12.637450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:12.637407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.336950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:13.336918    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:13.471783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:13.471752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.003924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:14.003886    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.145396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:14.144821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.281565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:14.281320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:14.972393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:14.972329    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:15.660961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:15.660890    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:16.448095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:16.448066    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:16.972336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:16.972304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.110464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:17.109715    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.242399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:17.242320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:17.567898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:17.559561    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.242457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:18.242401    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.359887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:18.359853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.522333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:18.522251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.706205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:18.706174    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:18.840210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:18.840126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.193973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:19.193943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.312168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:19.312126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.444482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:19.444325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:19.563638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:19.563609    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.231902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:20.231867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.390008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:20.389976    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:20.755124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:20.755092    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:21.858530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:21.858501    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:22.513212 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:23:22.513176    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:23:22.816676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:22.816583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.166390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:23.166359    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.288524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:23.288491    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:23.819453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:23.819402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:24.359704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:24.359652    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:24.532275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:24.532236    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:25.054830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:25.054770    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:25.741260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:25.741228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:26.548324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:26.548289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:27.460981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:27.460948    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.366348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:28.366311    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:28.499527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:28.499490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:29.260738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:29.260703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.133743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.133707    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.396503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.396377    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:23:30.411494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.411466    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:23:30.549194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.549164    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.673512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.672543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:30.932887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:30.932798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:31.600017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:31.599928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:32.350335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:32.350296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:32.482817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:32.481474    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:33.182293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:33.182266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:33.847350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:33.847317    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:23:33.847609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:33.847579    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" podUID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" containerName="maas-api" containerID="cri-o://6883ebf39c4e5183923a76db7d63680acaa9b49af0d94e46353980b20c771330" gracePeriod=30
Oct 30 21:23:33.933915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:33.933879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.012458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.012362    6738 generic.go:356] "Generic (PLEG): container finished" podID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" containerID="6883ebf39c4e5183923a76db7d63680acaa9b49af0d94e46353980b20c771330" exitCode=0
Oct 30 21:23:34.012458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.012426    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" event={"ID":"984f011c-adf9-4e70-b46c-bcbf79ad0b0a","Type":"ContainerDied","Data":"6883ebf39c4e5183923a76db7d63680acaa9b49af0d94e46353980b20c771330"}
Oct 30 21:23:34.178459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.171417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.355138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.353732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.480671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.480639    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:23:34.560979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.560944    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\" (UniqueName: \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\") pod \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") "
Oct 30 21:23:34.561164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.561052    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\") pod \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") "
Oct 30 21:23:34.561164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.561131    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") pod \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") "
Oct 30 21:23:34.561298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.561177    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\") pod \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\" (UID: \"984f011c-adf9-4e70-b46c-bcbf79ad0b0a\") "
Oct 30 21:23:34.561815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.561782    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca" pod "984f011c-adf9-4e70-b46c-bcbf79ad0b0a" (UID: "984f011c-adf9-4e70-b46c-bcbf79ad0b0a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:34.564111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.564084    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb" pod "984f011c-adf9-4e70-b46c-bcbf79ad0b0a" (UID: "984f011c-adf9-4e70-b46c-bcbf79ad0b0a"). InnerVolumeSpecName "kube-api-access-775qb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:34.564233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.564132    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs" pod "984f011c-adf9-4e70-b46c-bcbf79ad0b0a" (UID: "984f011c-adf9-4e70-b46c-bcbf79ad0b0a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:23:34.565588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.565562    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls" pod "984f011c-adf9-4e70-b46c-bcbf79ad0b0a" (UID: "984f011c-adf9-4e70-b46c-bcbf79ad0b0a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:23:34.569635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.569603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:34.663784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.663735    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-775qb\" (UniqueName: \"kubernetes.io/projected/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-kube-api-access-775qb\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:34.663784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.663781    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:34.663784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.663791    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:34.664041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.663800    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/984f011c-adf9-4e70-b46c-bcbf79ad0b0a-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:34.929305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:34.929270    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.021986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.021957    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"
Oct 30 21:23:35.022180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.021988    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8" event={"ID":"984f011c-adf9-4e70-b46c-bcbf79ad0b0a","Type":"ContainerDied","Data":"657a1ea54409e1734a780f0e99d9f9f6f5dfdceff9a9aada1ecb71bf7827b040"}
Oct 30 21:23:35.022180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.022055    6738 scope.go:122] "RemoveContainer" containerID="6883ebf39c4e5183923a76db7d63680acaa9b49af0d94e46353980b20c771330"
Oct 30 21:23:35.050282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.050259    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:23:35.056750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.056729    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-80e936-77cbfb9f88-7qpq8"]
Oct 30 21:23:35.174428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.174396    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" path="/var/lib/kubelet/pods/984f011c-adf9-4e70-b46c-bcbf79ad0b0a/volumes"
Oct 30 21:23:35.397315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.397277    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" podUID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" containerName="payload-processing" containerID="cri-o://20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3" gracePeriod=25
Oct 30 21:23:35.412533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.412492    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" podUID="b31a1b55-93f7-4316-b95a-58b16decb7e9" containerName="payload-pre-processing" containerID="cri-o://2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9" gracePeriod=25
Oct 30 21:23:35.496385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.496359    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:35.867426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.864697    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:23:35.908458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:35.900740    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:23:36.002235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002159    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\") pod \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") "
Oct 30 21:23:36.002235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002211    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\") pod \"b31a1b55-93f7-4316-b95a-58b16decb7e9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") "
Oct 30 21:23:36.002509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002373    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\" (UniqueName: \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\") pod \"b31a1b55-93f7-4316-b95a-58b16decb7e9\" (UID: \"b31a1b55-93f7-4316-b95a-58b16decb7e9\") "
Oct 30 21:23:36.002509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002497    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\" (UniqueName: \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\") pod \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\" (UID: \"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e\") "
Oct 30 21:23:36.002633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002606    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume" pod "1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" (UID: "1b2a166f-5fa2-4696-9ed2-7758b2f3b44e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:36.002690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.002669    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume" pod "b31a1b55-93f7-4316-b95a-58b16decb7e9" (UID: "b31a1b55-93f7-4316-b95a-58b16decb7e9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:23:36.003171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.003148    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:36.003257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.003173    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b31a1b55-93f7-4316-b95a-58b16decb7e9-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:36.004863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.004839    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm" pod "b31a1b55-93f7-4316-b95a-58b16decb7e9" (UID: "b31a1b55-93f7-4316-b95a-58b16decb7e9"). InnerVolumeSpecName "kube-api-access-kg8wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:36.004952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.004884    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78" pod "1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" (UID: "1b2a166f-5fa2-4696-9ed2-7758b2f3b44e"). InnerVolumeSpecName "kube-api-access-sxb78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:23:36.033156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.033131    6738 generic.go:356] "Generic (PLEG): container finished" podID="b31a1b55-93f7-4316-b95a-58b16decb7e9" containerID="2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9" exitCode=0
Oct 30 21:23:36.033254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.033178    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"
Oct 30 21:23:36.033254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.033209    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" event={"ID":"b31a1b55-93f7-4316-b95a-58b16decb7e9","Type":"ContainerDied","Data":"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"}
Oct 30 21:23:36.033354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.033262    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9" event={"ID":"b31a1b55-93f7-4316-b95a-58b16decb7e9","Type":"ContainerDied","Data":"7a4a0d1a0cc234d30d7d5f0d4ca9bd40cf5f64d1790b6bb8587604f9d005d7b8"}
Oct 30 21:23:36.033354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.033290    6738 scope.go:122] "RemoveContainer" containerID="2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"
Oct 30 21:23:36.034802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.034778    6738 generic.go:356] "Generic (PLEG): container finished" podID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" containerID="20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3" exitCode=0
Oct 30 21:23:36.034882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.034823    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"
Oct 30 21:23:36.034979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.034963    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" event={"ID":"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e","Type":"ContainerDied","Data":"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"}
Oct 30 21:23:36.035027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.035007    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr" event={"ID":"1b2a166f-5fa2-4696-9ed2-7758b2f3b44e","Type":"ContainerDied","Data":"d2116abecf8aaa2af700eae893bbbe56f9ff13fe1b7c4885daff886af425db17"}
Oct 30 21:23:36.047351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.047333    6738 scope.go:122] "RemoveContainer" containerID="2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"
Oct 30 21:23:36.047672 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:23:36.047647    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9\": container with ID starting with 2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9 not found: ID does not exist" containerID="2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"
Oct 30 21:23:36.047790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.047681    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9"} err="failed to get container status \"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9\": rpc error: code = NotFound desc = could not find container \"2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9\": container with ID starting with 2e88d6aed3e16d22c1a7067f0b5c5266007a3213bd7ff9318a544f40e38074a9 not found: ID does not exist"
Oct 30 21:23:36.047790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.047705    6738 scope.go:122] "RemoveContainer" containerID="20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"
Oct 30 21:23:36.061780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.061755    6738 scope.go:122] "RemoveContainer" containerID="20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"
Oct 30 21:23:36.062121 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:23:36.062101    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3\": container with ID starting with 20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3 not found: ID does not exist" containerID="20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"
Oct 30 21:23:36.062233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.062127    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3"} err="failed to get container status \"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3\": rpc error: code = NotFound desc = could not find container \"20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3\": container with ID starting with 20a365f300a44e8f459cb9f7a979b4a6c57cbbe7213f19f9922269be7a1800f3 not found: ID does not exist"
Oct 30 21:23:36.067093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.067068    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:23:36.071774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.071750    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-80e936-9b964bf9-9q7d9"]
Oct 30 21:23:36.084959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.084936    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:23:36.089122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.089100    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-80e936-75f96d747f-nrclr"]
Oct 30 21:23:36.104814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.104791    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg8wm\" (UniqueName: \"kubernetes.io/projected/b31a1b55-93f7-4316-b95a-58b16decb7e9-kube-api-access-kg8wm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:36.104814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.104812    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxb78\" (UniqueName: \"kubernetes.io/projected/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e-kube-api-access-sxb78\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:23:36.554655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:36.554626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:37.165504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:37.165475    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" path="/var/lib/kubelet/pods/1b2a166f-5fa2-4696-9ed2-7758b2f3b44e/volumes"
Oct 30 21:23:37.165913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:37.165900    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b31a1b55-93f7-4316-b95a-58b16decb7e9" path="/var/lib/kubelet/pods/b31a1b55-93f7-4316-b95a-58b16decb7e9/volumes"
Oct 30 21:23:37.337220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:37.337187    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:38.046617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:38.046582    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:38.947499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:38.946324    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:39.641933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:39.641901    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:40.356054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:40.356023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:41.137100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:41.137068    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:41.953894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:41.953864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.155497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:42.155341    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.277245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:42.277156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.416662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:42.416632    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.672778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:42.672749    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:42.939331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:42.939247    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.349166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:43.349083    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.466037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:43.466005    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.580207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:43.580171    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.845922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:43.845884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:43.960157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:43.960121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:44.717406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:44.716923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:45.062759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:45.062688    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:45.495679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:45.495627    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:45.896169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:45.895157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:46.141355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:46.141320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:46.828341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:46.828302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:47.853133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:47.853096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:48.474996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:48.474955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:48.605468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:48.602242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:48.867843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:48.867790    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.047651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:49.047616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.718677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:49.718641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:49.862818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:49.862786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:50.156648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:50.156610    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:51.117025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:51.116990    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:51.258060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:51.258031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:51.379949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:51.379719    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:52.034427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:52.034390    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:52.509510 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:23:52.509477    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:23:52.751094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:52.751063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:53.235359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:53.235332    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:53.376908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:53.376864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:53.854423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:53.854387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:54.800029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:54.799991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:55.452946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:55.452913    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:56.334978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:56.334946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:57.085370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:57.085338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:57.741549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:57.741498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:58.427266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:58.427223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:58.539494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:58.539416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:59.235937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:59.235875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:23:59.966402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:23:59.966364    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:00.644979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:00.644933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:01.533717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:01.533685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:02.252610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:02.252556    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:03.021740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:03.021692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:03.840625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:03.840583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.361503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.361470    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:24:04.362248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.362217    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" podUID="b8cd1623-f72c-4c94-880a-0249905f2aa2" containerName="maas-api" containerID="cri-o://859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619" gracePeriod=30
Oct 30 21:24:04.428201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.428167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.560253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.560201    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.705706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.705545    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.889229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.885168    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:04.940207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:04.939865    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:24:05.043150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.043077    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\") pod \"b8cd1623-f72c-4c94-880a-0249905f2aa2\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") "
Oct 30 21:24:05.043299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.043161    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\") pod \"b8cd1623-f72c-4c94-880a-0249905f2aa2\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") "
Oct 30 21:24:05.043299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.043226    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\" (UniqueName: \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\") pod \"b8cd1623-f72c-4c94-880a-0249905f2aa2\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") "
Oct 30 21:24:05.043299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.043270    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") pod \"b8cd1623-f72c-4c94-880a-0249905f2aa2\" (UID: \"b8cd1623-f72c-4c94-880a-0249905f2aa2\") "
Oct 30 21:24:05.044081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.043574    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca" pod "b8cd1623-f72c-4c94-880a-0249905f2aa2" (UID: "b8cd1623-f72c-4c94-880a-0249905f2aa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:05.044081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.044052    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8cd1623-f72c-4c94-880a-0249905f2aa2-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:05.045651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.045623    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls" pod "b8cd1623-f72c-4c94-880a-0249905f2aa2" (UID: "b8cd1623-f72c-4c94-880a-0249905f2aa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:05.045781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.045678    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2" pod "b8cd1623-f72c-4c94-880a-0249905f2aa2" (UID: "b8cd1623-f72c-4c94-880a-0249905f2aa2"). InnerVolumeSpecName "kube-api-access-bfnp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:05.045781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.045740    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs" pod "b8cd1623-f72c-4c94-880a-0249905f2aa2" (UID: "b8cd1623-f72c-4c94-880a-0249905f2aa2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:05.053804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.053780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:05.146204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.146167    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:05.146204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.146204    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfnp2\" (UniqueName: \"kubernetes.io/projected/b8cd1623-f72c-4c94-880a-0249905f2aa2-kube-api-access-bfnp2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:05.146204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.146219    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8cd1623-f72c-4c94-880a-0249905f2aa2-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:05.354002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.353919    6738 generic.go:356] "Generic (PLEG): container finished" podID="b8cd1623-f72c-4c94-880a-0249905f2aa2" containerID="859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619" exitCode=0
Oct 30 21:24:05.354002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.353962    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"
Oct 30 21:24:05.354223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.354010    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" event={"ID":"b8cd1623-f72c-4c94-880a-0249905f2aa2","Type":"ContainerDied","Data":"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"}
Oct 30 21:24:05.354223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.354068    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr" event={"ID":"b8cd1623-f72c-4c94-880a-0249905f2aa2","Type":"ContainerDied","Data":"5dd56482c817171cb0e1bf8a175ef3114b4d248271ca2a595eb1475b09705b8b"}
Oct 30 21:24:05.354223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.354100    6738 scope.go:122] "RemoveContainer" containerID="859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"
Oct 30 21:24:05.365702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.365466    6738 scope.go:122] "RemoveContainer" containerID="859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"
Oct 30 21:24:05.366003 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:05.365757    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619\": container with ID starting with 859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619 not found: ID does not exist" containerID="859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"
Oct 30 21:24:05.366003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.365791    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619"} err="failed to get container status \"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619\": rpc error: code = NotFound desc = could not find container \"859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619\": container with ID starting with 859af7d6d6a1f631c6ebe00840adf15778213296e9d5aae65145685fd1b4f619 not found: ID does not exist"
Oct 30 21:24:05.373146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.372139    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:24:05.373357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.373217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:05.376631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.376611    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-43f6ab-66c996c6c-288dr"]
Oct 30 21:24:05.794024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:05.793990    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:06.461821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:06.461781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.156615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:07.156582    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:07.163949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:07.163925    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8cd1623-f72c-4c94-880a-0249905f2aa2" path="/var/lib/kubelet/pods/b8cd1623-f72c-4c94-880a-0249905f2aa2/volumes"
Oct 30 21:24:08.129422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:08.129391    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:08.970340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:08.970310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.657193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:09.657157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:09.838106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:09.838067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.309955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:10.309919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.598477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:10.598366    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.764460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:10.764006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:10.891915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:10.891843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:11.661912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:11.661879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:11.953224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:11.953111    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:12.085308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:12.085270    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:12.323044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:12.322966    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.009068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:13.009036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:13.969926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:13.969401    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:14.340570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:14.340428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:14.833466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:14.826565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.092512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:15.088076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.211462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:15.210794    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.342462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:15.338967    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:15.881565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:15.881530    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:24:15.881854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:15.881825    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:24:16.236835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:16.236745    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.873267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:16.873235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:16.996845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:16.996814    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:17.300085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:17.300012    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.118843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:18.118807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.567063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:18.566899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:18.851805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:18.851696    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:19.013688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:19.013650    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:20.071503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:20.071428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:20.195668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:20.195626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:20.364526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:20.340328    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:21.249622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:21.249590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:21.636117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:21.636075    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:21.759022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:21.758992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.050369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:22.050288    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:22.509165 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:22.509126    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:24:22.762084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:22.762007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:23.441165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:23.441129    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:24.250063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:24.250034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:24.988494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:24.988424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:25.105291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:25.105258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:25.386254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:25.386206    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:25.934905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:25.934878    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:26.620302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:26.620266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:27.453134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:27.453099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:28.146083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.146033    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:24:28.158110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158083    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" containerName="maas-api"
Oct 30 21:24:28.158354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158338    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" containerName="maas-api"
Oct 30 21:24:28.158508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158496    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b31a1b55-93f7-4316-b95a-58b16decb7e9" containerName="payload-pre-processing"
Oct 30 21:24:28.158614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158602    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b31a1b55-93f7-4316-b95a-58b16decb7e9" containerName="payload-pre-processing"
Oct 30 21:24:28.158727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158718    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b8cd1623-f72c-4c94-880a-0249905f2aa2" containerName="maas-api"
Oct 30 21:24:28.158818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158808    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8cd1623-f72c-4c94-880a-0249905f2aa2" containerName="maas-api"
Oct 30 21:24:28.158979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.158967    6738 cpu_manager.go:414] "containerMap: removing container" podUID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" containerName="payload-processing"
Oct 30 21:24:28.159097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.159083    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" containerName="payload-processing"
Oct 30 21:24:28.159229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.159216    6738 cpu_manager.go:414] "containerMap: removing container" podUID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" containerName="maas-api"
Oct 30 21:24:28.159346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.159330    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" containerName="maas-api"
Oct 30 21:24:28.160401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.160079    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d4e61e8-d9c2-4bf2-999a-f28d3dc5ad2f" containerName="maas-api"
Oct 30 21:24:28.160576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.160561    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b31a1b55-93f7-4316-b95a-58b16decb7e9" containerName="payload-pre-processing"
Oct 30 21:24:28.160696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.160677    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8cd1623-f72c-4c94-880a-0249905f2aa2" containerName="maas-api"
Oct 30 21:24:28.160813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.160802    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b2a166f-5fa2-4696-9ed2-7758b2f3b44e" containerName="payload-processing"
Oct 30 21:24:28.160924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.160915    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="984f011c-adf9-4e70-b46c-bcbf79ad0b0a" containerName="maas-api"
Oct 30 21:24:28.174129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.174104    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.176276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.176248    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:24:28.181717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.181685    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-7db332-serving-cert\""
Oct 30 21:24:28.219054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.219017    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:28.277461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.274409    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.277461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.274578    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-954bg\" (UniqueName: \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.277461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.274663    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.277461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.274708    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.376036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.376000    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.376206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.376131    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.376294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.376209    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-954bg\" (UniqueName: \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.376294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.376272    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.376398 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:28.376290    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-serving-cert: secret "maas-api-e2e-praxis-a-7db332-serving-cert" not found
Oct 30 21:24:28.376398 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:28.376378    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls podName:787d1502-7802-4075-ab2b-152d7dfb5d34 nodeName:}" failed. No retries permitted until 2026-10-30 21:24:28.876357896 +0000 UTC m=+1325.832045105 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls") pod "maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" (UID: "787d1502-7802-4075-ab2b-152d7dfb5d34") : secret "maas-api-e2e-praxis-a-7db332-serving-cert" not found
Oct 30 21:24:28.376767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.376749    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.378952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.378933    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.384523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.384499    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-954bg\" (UniqueName: \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.387007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.386982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:28.590535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.590377    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:28.905899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.905830    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.912857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.912828    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") pod \"maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:28.989985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:28.989945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.096730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.096700    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:29.109277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.109250    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.267511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.266348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.448375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.448336    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.573750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.573662    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.625299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.625270    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:24:29.701080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.701051    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:29.922426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.922384    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" event={"ID":"787d1502-7802-4075-ab2b-152d7dfb5d34","Type":"ContainerStarted","Data":"1e9fd69f863ecdcababb588daf65755f6cfc4671b081b698c67bcfee49bcb60a"}
Oct 30 21:24:29.985935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:29.985894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:30.427134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:30.427097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:30.931811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:30.931775    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" event={"ID":"787d1502-7802-4075-ab2b-152d7dfb5d34","Type":"ContainerStarted","Data":"bf5206d7e60bdbba16f0785ca5ce37e1525bc3af0f59c9336a98eed7b5924a47"}
Oct 30 21:24:30.932023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:30.931973    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:30.952605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:30.952559    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" podStartSLOduration=2.55672044 podStartE2EDuration="2.952544878s" totalImagesPullingTime="395.824438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:30.950954747 +0000 UTC m=+1327.906641977" watchObservedRunningTime="2026-10-30 21:24:30.952544878 +0000 UTC m=+1327.908232105"
Oct 30 21:24:31.257329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:31.257251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.056608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:32.056574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:32.872199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:32.872170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:33.652131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:33.652098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:34.535758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:34.535705    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:35.329793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:35.329760    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.005695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:36.005660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.265320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:36.265218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.479382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:36.479349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:36.944720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:36.944695    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:24:37.393303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:37.393262    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:38.325305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:38.325269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:39.504754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:39.504719    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.286013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:40.285979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.445460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:40.438220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:40.632061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:40.632030    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:41.110488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:41.110360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:42.053880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:42.053850    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:42.735348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:42.735313    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:43.498488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:43.498403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.035072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:44.035041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:44.169733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:44.169699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.236929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:45.236893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.353204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:45.353169    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:45.743526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:45.743491    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.248313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:46.248267    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.371088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:46.371053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.884231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:46.884179    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:46.994140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:46.994096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.157419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.157284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.245104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.239894    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:24:47.245104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.240131    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" podUID="869444b0-ff99-4315-afdc-69e7ca5c658d" containerName="maas-api" containerID="cri-o://27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0" gracePeriod=30
Oct 30 21:24:47.291383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.291343    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.418565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.413195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.532460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.530856    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.686149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.685563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.946967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.946738    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:47.950230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:47.950019    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:24:48.058877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.054709    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") pod \"869444b0-ff99-4315-afdc-69e7ca5c658d\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") "
Oct 30 21:24:48.058877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.054930    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\") pod \"869444b0-ff99-4315-afdc-69e7ca5c658d\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") "
Oct 30 21:24:48.058877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.054986    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\" (UniqueName: \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\") pod \"869444b0-ff99-4315-afdc-69e7ca5c658d\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") "
Oct 30 21:24:48.058877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.055025    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\") pod \"869444b0-ff99-4315-afdc-69e7ca5c658d\" (UID: \"869444b0-ff99-4315-afdc-69e7ca5c658d\") "
Oct 30 21:24:48.058877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.056275    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca" pod "869444b0-ff99-4315-afdc-69e7ca5c658d" (UID: "869444b0-ff99-4315-afdc-69e7ca5c658d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:24:48.064513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.064452    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls" pod "869444b0-ff99-4315-afdc-69e7ca5c658d" (UID: "869444b0-ff99-4315-afdc-69e7ca5c658d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:48.066725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.066687    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d" pod "869444b0-ff99-4315-afdc-69e7ca5c658d" (UID: "869444b0-ff99-4315-afdc-69e7ca5c658d"). InnerVolumeSpecName "kube-api-access-kp69d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:24:48.067665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.067629    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs" pod "869444b0-ff99-4315-afdc-69e7ca5c658d" (UID: "869444b0-ff99-4315-afdc-69e7ca5c658d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:24:48.135466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.135415    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:48.158824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.158788    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:48.159067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.158836    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp69d\" (UniqueName: \"kubernetes.io/projected/869444b0-ff99-4315-afdc-69e7ca5c658d-kube-api-access-kp69d\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:48.159067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.158855    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/869444b0-ff99-4315-afdc-69e7ca5c658d-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:48.159200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.159174    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/869444b0-ff99-4315-afdc-69e7ca5c658d-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:24:48.271082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.270997    6738 generic.go:356] "Generic (PLEG): container finished" podID="869444b0-ff99-4315-afdc-69e7ca5c658d" containerID="27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0" exitCode=0
Oct 30 21:24:48.271082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.271063    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"
Oct 30 21:24:48.271315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.271074    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" event={"ID":"869444b0-ff99-4315-afdc-69e7ca5c658d","Type":"ContainerDied","Data":"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"}
Oct 30 21:24:48.271315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.271163    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx" event={"ID":"869444b0-ff99-4315-afdc-69e7ca5c658d","Type":"ContainerDied","Data":"0a25207ef594116ad28cd8f4cf06ad0f0866070ca02676a1f735fe2dc098885e"}
Oct 30 21:24:48.271315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.271187    6738 scope.go:122] "RemoveContainer" containerID="27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"
Oct 30 21:24:48.282503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.282489    6738 scope.go:122] "RemoveContainer" containerID="27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"
Oct 30 21:24:48.282760 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:48.282742    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0\": container with ID starting with 27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0 not found: ID does not exist" containerID="27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"
Oct 30 21:24:48.282827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.282767    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0"} err="failed to get container status \"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0\": rpc error: code = NotFound desc = could not find container \"27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0\": container with ID starting with 27f712a8276a34b544418203b853184e15e7a008c5cd45feddc92f4e70a43dd0 not found: ID does not exist"
Oct 30 21:24:48.297654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.297610    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:24:48.302649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:48.302616    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-6563b6-76d4cdc95d-8xctx"]
Oct 30 21:24:49.044223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:49.044189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:49.167228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:49.167194    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="869444b0-ff99-4315-afdc-69e7ca5c658d" path="/var/lib/kubelet/pods/869444b0-ff99-4315-afdc-69e7ca5c658d/volumes"
Oct 30 21:24:49.824815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:49.824780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.185864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:50.185831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.455236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:50.455156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:50.800153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:50.800063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.028461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.018292    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.244910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.244879    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.354921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.354885    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.584194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.584109    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.732463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.719314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:51.851166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:51.851078    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:52.346411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:52.346379    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:52.509820 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:24:52.509787    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:24:53.128267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:53.128226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:53.252010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:53.251983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:53.820161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:53.820130    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:54.605164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:54.605131    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:55.502959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:55.502923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:56.119200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:56.119170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.049333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.049295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.786150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.786119    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:24:57.787541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.787516    6738 cpu_manager.go:414] "containerMap: removing container" podUID="869444b0-ff99-4315-afdc-69e7ca5c658d" containerName="maas-api"
Oct 30 21:24:57.787686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.787670    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="869444b0-ff99-4315-afdc-69e7ca5c658d" containerName="maas-api"
Oct 30 21:24:57.787837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.787824    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="869444b0-ff99-4315-afdc-69e7ca5c658d" containerName="maas-api"
Oct 30 21:24:57.796336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.796312    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.803046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.801316    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-67b92c-serving-cert\""
Oct 30 21:24:57.803046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.802727    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:24:57.838725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.838697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.861134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.861096    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.861271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.861152    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.861271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.861199    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.861397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.861314    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9cj\" (UniqueName: \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.951337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.951304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:57.966525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.963937    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.966525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.963984    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.966525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.964026    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.966525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.964143    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9cj\" (UniqueName: \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.969017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.968994    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.971840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.971816    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.972752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.972697    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:57.986453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:57.984687    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9cj\" (UniqueName: \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\") pod \"maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:58.120674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.120640    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:58.413597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.412791    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:24:58.415673 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:24:58.414855    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbacc06c_ed14_44e0_a916_a40796ca7622.slice/crio-2c18d5fbaf9e3a53f6d7211c75f53434c383c1f0638328b0e1033ffc6578f603 WatchSource:0}: Error finding container 2c18d5fbaf9e3a53f6d7211c75f53434c383c1f0638328b0e1033ffc6578f603: Status 404 returned error can't find the container with id 2c18d5fbaf9e3a53f6d7211c75f53434c383c1f0638328b0e1033ffc6578f603
Oct 30 21:24:58.452800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.452756    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.603893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.603854    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.822059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.821984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:58.968696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:58.968612    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:24:59.353980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:59.353913    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" event={"ID":"bbacc06c-ed14-44e0-a916-a40796ca7622","Type":"ContainerStarted","Data":"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"}
Oct 30 21:24:59.353980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:59.353981    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" event={"ID":"bbacc06c-ed14-44e0-a916-a40796ca7622","Type":"ContainerStarted","Data":"2c18d5fbaf9e3a53f6d7211c75f53434c383c1f0638328b0e1033ffc6578f603"}
Oct 30 21:24:59.354488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:59.354128    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:24:59.370515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:59.370196    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" podStartSLOduration=1.945319072 podStartE2EDuration="2.370180591s" totalImagesPullingTime="424.861519ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:24:59.3697136 +0000 UTC m=+1356.325400829" watchObservedRunningTime="2026-10-30 21:24:59.370180591 +0000 UTC m=+1356.325867822"
Oct 30 21:24:59.527676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:24:59.527644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:00.359452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:00.359399    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:01.219181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:01.219146    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:02.073234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:02.073183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:02.899755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:02.899712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:03.834370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:03.834340    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:05.254534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:05.254409    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:05.370313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:05.370236    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:25:06.074561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:06.074532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.223201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:07.223173    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:07.943779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:07.943751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:08.770706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:08.770668    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:09.521430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:09.521399    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.150604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:10.150573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:10.273006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:10.272974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.011712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:11.009626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:11.862838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:11.862810    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:12.024621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:12.024585    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:12.747951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:12.747921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:13.568870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:13.568820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:14.537954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:14.537924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:15.275119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:15.275081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:16.503852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:16.503767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.146226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:18.146193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.619719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:18.619691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:18.832444 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:18.832410    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:19.637481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:19.637421    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:20.443492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:20.443381    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.155731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:21.155690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.716515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:21.716484    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:21.848772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:21.848740    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:22.005307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:22.005206    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:22.514338 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:25:22.514306    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:25:22.830859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:22.830753    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.432150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:23.431962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.573828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:23.573784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:23.795341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:23.795309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:24.069915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:24.069827    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:24.247302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:24.247267    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:24.461072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:24.461041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:24.973594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:24.973548    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:25.738870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:25.738841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:26.409055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:26.409017    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.060812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.060764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.597886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.597853    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:25:27.598354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.598130    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" podUID="787d1502-7802-4075-ab2b-152d7dfb5d34" containerName="maas-api" containerID="cri-o://bf5206d7e60bdbba16f0785ca5ce37e1525bc3af0f59c9336a98eed7b5924a47" gracePeriod=30
Oct 30 21:25:27.659428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.659314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.761819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.761773    6738 generic.go:356] "Generic (PLEG): container finished" podID="787d1502-7802-4075-ab2b-152d7dfb5d34" containerID="bf5206d7e60bdbba16f0785ca5ce37e1525bc3af0f59c9336a98eed7b5924a47" exitCode=0
Oct 30 21:25:27.761819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.761821    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" event={"ID":"787d1502-7802-4075-ab2b-152d7dfb5d34","Type":"ContainerDied","Data":"bf5206d7e60bdbba16f0785ca5ce37e1525bc3af0f59c9336a98eed7b5924a47"}
Oct 30 21:25:27.849266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.849164    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:27.948874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.948847    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:25:27.954412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:27.954390    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:28.064030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.063993    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") pod \"787d1502-7802-4075-ab2b-152d7dfb5d34\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") "
Oct 30 21:25:28.064227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.064094    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\") pod \"787d1502-7802-4075-ab2b-152d7dfb5d34\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") "
Oct 30 21:25:28.064645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.064620    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\") pod \"787d1502-7802-4075-ab2b-152d7dfb5d34\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") "
Oct 30 21:25:28.064772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.064742    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\" (UniqueName: \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\") pod \"787d1502-7802-4075-ab2b-152d7dfb5d34\" (UID: \"787d1502-7802-4075-ab2b-152d7dfb5d34\") "
Oct 30 21:25:28.066053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.065971    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca" pod "787d1502-7802-4075-ab2b-152d7dfb5d34" (UID: "787d1502-7802-4075-ab2b-152d7dfb5d34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:28.068265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.068237    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls" pod "787d1502-7802-4075-ab2b-152d7dfb5d34" (UID: "787d1502-7802-4075-ab2b-152d7dfb5d34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:25:28.068386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.068298    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg" pod "787d1502-7802-4075-ab2b-152d7dfb5d34" (UID: "787d1502-7802-4075-ab2b-152d7dfb5d34"). InnerVolumeSpecName "kube-api-access-954bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:28.068466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.068452    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs" pod "787d1502-7802-4075-ab2b-152d7dfb5d34" (UID: "787d1502-7802-4075-ab2b-152d7dfb5d34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:25:28.167011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.166978    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787d1502-7802-4075-ab2b-152d7dfb5d34-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.167011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.167006    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-954bg\" (UniqueName: \"kubernetes.io/projected/787d1502-7802-4075-ab2b-152d7dfb5d34-kube-api-access-954bg\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.167011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.167018    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.167247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.167027    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787d1502-7802-4075-ab2b-152d7dfb5d34-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:28.435091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.434999    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:28.770039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.769975    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"
Oct 30 21:25:28.770039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.769990    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5" event={"ID":"787d1502-7802-4075-ab2b-152d7dfb5d34","Type":"ContainerDied","Data":"1e9fd69f863ecdcababb588daf65755f6cfc4671b081b698c67bcfee49bcb60a"}
Oct 30 21:25:28.770536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.770048    6738 scope.go:122] "RemoveContainer" containerID="bf5206d7e60bdbba16f0785ca5ce37e1525bc3af0f59c9336a98eed7b5924a47"
Oct 30 21:25:28.794412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.794387    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:25:28.798890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.798867    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-7db332-55f6cd68b5-hhts5"]
Oct 30 21:25:28.812731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:28.812710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:29.165147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:29.165118    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="787d1502-7802-4075-ab2b-152d7dfb5d34" path="/var/lib/kubelet/pods/787d1502-7802-4075-ab2b-152d7dfb5d34/volumes"
Oct 30 21:25:29.706948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:29.706891    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:29.974388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:29.974300    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:30.548797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:30.548764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:31.503338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:31.503303    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:31.789537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:31.789458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:32.223261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:32.223232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:32.346097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:32.346054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:32.621370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:32.621337    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:32.763762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:32.763705    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:33.925283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:33.922914    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:34.799725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:34.799690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.374823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:35.374756    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.501671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:35.501635    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.728493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:35.728339    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:35.878376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:35.876871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:36.711025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:36.710983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:37.752671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:37.752635    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.493983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:38.493942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.622568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:38.622525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:38.883985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:38.883946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:39.042796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:39.042712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:40.001993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:40.001955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:40.733515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:40.733480    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:41.543958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:41.543894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:42.378429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:42.378400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:43.401545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:43.401502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:44.179564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:44.178259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:45.584508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:45.584468    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.361413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:46.361376    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:46.476416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:46.476385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:47.146420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:47.146386    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:47.909725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:47.909690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:48.619831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:48.619784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.193932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:49.193893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.310678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:49.310556    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.441677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:49.441638    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.593968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:49.593892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:49.758134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:49.758080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.141636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.141568    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.335791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.335758    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:25:50.336025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.335984    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerName="maas-api" containerID="cri-o://ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693" gracePeriod=30
Oct 30 21:25:50.363975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.362085    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.84:8443/health\": dial tcp 10.134.0.84:8443: connect: connection refused"
Oct 30 21:25:50.390454 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.386790    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.579197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.564027    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.738960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.738852    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.852786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.852759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:50.867870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.867061    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:25:50.927235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.925742    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\") pod \"bbacc06c-ed14-44e0-a916-a40796ca7622\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") "
Oct 30 21:25:50.927235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.925822    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\") pod \"bbacc06c-ed14-44e0-a916-a40796ca7622\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") "
Oct 30 21:25:50.927235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.925940    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\") pod \"bbacc06c-ed14-44e0-a916-a40796ca7622\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") "
Oct 30 21:25:50.927235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.926043    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\" (UniqueName: \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\") pod \"bbacc06c-ed14-44e0-a916-a40796ca7622\" (UID: \"bbacc06c-ed14-44e0-a916-a40796ca7622\") "
Oct 30 21:25:50.927783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.927753    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca" pod "bbacc06c-ed14-44e0-a916-a40796ca7622" (UID: "bbacc06c-ed14-44e0-a916-a40796ca7622"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:25:50.931016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.930980    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj" pod "bbacc06c-ed14-44e0-a916-a40796ca7622" (UID: "bbacc06c-ed14-44e0-a916-a40796ca7622"). InnerVolumeSpecName "kube-api-access-4l9cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:25:50.939883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.939494    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs" pod "bbacc06c-ed14-44e0-a916-a40796ca7622" (UID: "bbacc06c-ed14-44e0-a916-a40796ca7622"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:25:50.939883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:50.939567    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls" pod "bbacc06c-ed14-44e0-a916-a40796ca7622" (UID: "bbacc06c-ed14-44e0-a916-a40796ca7622"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:25:51.028634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.028548    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbacc06c-ed14-44e0-a916-a40796ca7622-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:51.028634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.028573    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:51.028634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.028582    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbacc06c-ed14-44e0-a916-a40796ca7622-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:51.028634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.028590    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l9cj\" (UniqueName: \"kubernetes.io/projected/bbacc06c-ed14-44e0-a916-a40796ca7622-kube-api-access-4l9cj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:25:51.139096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.139059    6738 generic.go:356] "Generic (PLEG): container finished" podID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerID="ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693" exitCode=0
Oct 30 21:25:51.139261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.139111    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"
Oct 30 21:25:51.139261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.139132    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" event={"ID":"bbacc06c-ed14-44e0-a916-a40796ca7622","Type":"ContainerDied","Data":"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"}
Oct 30 21:25:51.139261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.139190    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln" event={"ID":"bbacc06c-ed14-44e0-a916-a40796ca7622","Type":"ContainerDied","Data":"2c18d5fbaf9e3a53f6d7211c75f53434c383c1f0638328b0e1033ffc6578f603"}
Oct 30 21:25:51.139261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.139203    6738 scope.go:122] "RemoveContainer" containerID="ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"
Oct 30 21:25:51.150036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.150009    6738 scope.go:122] "RemoveContainer" containerID="ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"
Oct 30 21:25:51.150284 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:25:51.150265    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693\": container with ID starting with ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693 not found: ID does not exist" containerID="ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"
Oct 30 21:25:51.150395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.150290    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693"} err="failed to get container status \"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693\": rpc error: code = NotFound desc = could not find container \"ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693\": container with ID starting with ef8dd063214088e8282642c38427e7a241fd036a5dbf095a290db4873dc30693 not found: ID does not exist"
Oct 30 21:25:51.168187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.168166    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:25:51.168308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.168205    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-67b92c-74f4bbf558-2rvln"]
Oct 30 21:25:51.259303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.259273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:51.728161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:51.728080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:52.508448 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:25:52.508402    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:25:52.636502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:52.636458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:53.167719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:53.167681    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" path="/var/lib/kubelet/pods/bbacc06c-ed14-44e0-a916-a40796ca7622/volumes"
Oct 30 21:25:53.448988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:53.448904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.140468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:54.140403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.457851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:54.457779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:54.568717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:54.568687    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:55.039741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:55.039702    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:55.759506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:55.759393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:56.449325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:56.449297    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:56.772545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:56.772460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:56.895716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:56.895673    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.122042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:57.122006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.501316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:57.501235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.745318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:57.745290    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:57.875927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:57.875900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:58.100324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:58.100293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:58.522471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:58.522422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.108724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:59.108690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.280929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:59.280895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.426098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:59.426052    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.571089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:59.571049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:25:59.964979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:25:59.964946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:00.987149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:00.987119    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:02.681404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:02.681361    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:02.815577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:02.815525    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:26:02.960517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:02.934821    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:03.023486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:02.992776    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:03.138463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:03.134768    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:03.234637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:03.234543    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:03.315580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:03.315527    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:03.449869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:03.449819    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:04.239002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:04.238953    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.122580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:05.122548    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:05.864352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:05.864316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.157856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:06.157777    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.275428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:06.275400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.521547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:06.521422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:06.834793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:06.834694    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:07.664205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:07.664174    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:08.678668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:08.678634    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:09.499174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:09.499147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.251909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:10.251880    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.453653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:10.453619    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:10.847573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:10.847535    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.333276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:11.333247    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:11.507016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:11.506954    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:12.239856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:12.239824    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:13.037235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:13.037200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.051905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:14.051871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.704744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:14.704709    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:14.873542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:14.873511    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.055092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:15.055009    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.815460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:15.815324    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:15.994521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:15.994490    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:26:15.994716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:15.994694    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy" containerID="cri-o://2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0" gracePeriod=30
Oct 30 21:26:16.030937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:16.030893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:16.232462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:16.224021    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:16.959393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:16.959359    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:17.852920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:17.852888    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:18.675175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:18.675138    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:18.806412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:18.805002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:19.748553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:19.748467    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:19.898982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:19.898932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:20.006157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:20.006077    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:20.654561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:20.654528    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:21.286724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.286701    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:26:21.311142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311030    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311112    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311219    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311263    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311331    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311389    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.311814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.311722    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:21.312279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.312246    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:21.312513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.312487    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:21.313124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.313096    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.313218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.313201    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.313344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.313326    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\") pod \"097310d5-1f77-46d6-bd14-dbfafc5d4019\" (UID: \"097310d5-1f77-46d6-bd14-dbfafc5d4019\") "
Oct 30 21:26:21.314166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.314130    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.314302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.314286    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.314390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.314379    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.315052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.315019    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:26:21.315386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.315366    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:21.320050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.318489    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:21.320050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.318742    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "kube-api-access-8kffs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:26:21.320050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.318884    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:26:21.320050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.319957    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy" pod "097310d5-1f77-46d6-bd14-dbfafc5d4019" (UID: "097310d5-1f77-46d6-bd14-dbfafc5d4019"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.416612    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.424286    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kffs\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-kube-api-access-8kffs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.424306    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.424320    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/097310d5-1f77-46d6-bd14-dbfafc5d4019-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.424351    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.424390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.424366    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/097310d5-1f77-46d6-bd14-dbfafc5d4019-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:26:21.526314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.526232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:21.620627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.620589    6738 generic.go:356] "Generic (PLEG): container finished" podID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerID="2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0" exitCode=0
Oct 30 21:26:21.620870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.620846    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" event={"ID":"097310d5-1f77-46d6-bd14-dbfafc5d4019","Type":"ContainerDied","Data":"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"}
Oct 30 21:26:21.620948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.620915    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v" event={"ID":"097310d5-1f77-46d6-bd14-dbfafc5d4019","Type":"ContainerDied","Data":"98feed5577713334648317bef8bd6abb93901626f300bdec9877da34c68ace87"}
Oct 30 21:26:21.621003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.620951    6738 scope.go:122] "RemoveContainer" containerID="2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"
Oct 30 21:26:21.621159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.621142    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"
Oct 30 21:26:21.646053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.645033    6738 scope.go:122] "RemoveContainer" containerID="2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"
Oct 30 21:26:21.646053 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:26:21.645452    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0\": container with ID starting with 2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0 not found: ID does not exist" containerID="2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"
Oct 30 21:26:21.646053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.645494    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0"} err="failed to get container status \"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0\": rpc error: code = NotFound desc = could not find container \"2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0\": container with ID starting with 2beb50639bc73522bef7c24df1657210f764bc9c36f2f3e08b41aec58c18bed0 not found: ID does not exist"
Oct 30 21:26:21.663849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.663820    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:26:21.666837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:21.666812    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-ct65v"]
Oct 30 21:26:22.619462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:22.618941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:22.650170 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:26:22.650133    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:26:22.806465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:22.806182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:23.581612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:23.581571    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" path="/var/lib/kubelet/pods/097310d5-1f77-46d6-bd14-dbfafc5d4019/volumes"
Oct 30 21:26:23.801546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:23.801503    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.015384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.004742    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:26:24.134732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.133543    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.194704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.182885    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.383462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.382525    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:26:24.407463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.403528    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.639714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.639670    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.673028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.672976    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:26:24.816461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.815038    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:24.816461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.815817    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:26:24.838775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:24.838736    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.274852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.229178    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:26:25.308325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.276663    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:26:25.330578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.280376    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.391461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.308629    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:26:25.391461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.374586    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.432461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.427537    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:26:25.432461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.427543    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.489409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.489372    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:25.569547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.542217    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:26:25.692450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.687695    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:26:25.692450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.687769    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:26:25.692450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.687806    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:26:25.692450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.687842    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:26:25.692450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.664205    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.720462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.673572    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:26:25.747408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.727380    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:26:25.747408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.728949    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:26:25.839682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.839580    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:25.850648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.841672    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:26:25.878463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:25.852137    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:26.315584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.315531    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:26:26.342855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.321807    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:26.354943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.337990    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:26:26.355247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.338209    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Oct 30 21:26:26.429598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.429549    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:26:26.511472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.429570    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Oct 30 21:26:26.910482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.701762    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:26.910482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.758951    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:26.910482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:26.843886    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:26:27.144145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.144103    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": context deadline exceeded"
Oct 30 21:26:27.551813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.416530    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:26:27.551813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.478051    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Oct 30 21:26:27.551813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.511755    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded"
Oct 30 21:26:27.567830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.567805    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:26:27.568064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.568053    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:26:27.568201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.568192    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:26:27.568320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.568312    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:26:27.568464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.568452    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:26:27.584460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.583672    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:26:27.587479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.585180    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:26:27.625774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.600566    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:26:27.658725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.658696    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:26:27.664833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.664805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:27.814125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:27.814023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.302214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:28.302180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.476461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:28.472399    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:28.860077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:28.860047    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:29.981193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:29.981162    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:31.012039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:31.012006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:31.514777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:31.514743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.051970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:32.051933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.238819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:32.238786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:32.777768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:32.777736    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:33.197562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:33.197526    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:33.400580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:33.400539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:33.685247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:33.685213    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.701476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:34.701227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:34.952650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:34.952563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.219236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:35.217619    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.651233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:35.651150    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:35.902163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:35.902086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.052395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:36.052363    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.469825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:36.469794    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.607695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:36.607654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:36.723009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:36.722921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.167095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:37.167057    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.822381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:37.822335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:37.972024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:37.971984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:38.603576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:38.603541    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.008619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:39.008593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.180755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:39.180720    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.539343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:39.538805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:39.704065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:39.704000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:40.726790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:40.726748    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:41.434576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:41.434541    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:41.687395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:41.687311    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:42.137084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:42.137041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:43.091843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:43.091805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.407092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:45.407059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:45.632962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:45.632803    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.120123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120085    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:26:46.120883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120860    6738 cpu_manager.go:414] "containerMap: removing container" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy"
Oct 30 21:26:46.120883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120883    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy"
Oct 30 21:26:46.121063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120905    6738 cpu_manager.go:414] "containerMap: removing container" podUID="787d1502-7802-4075-ab2b-152d7dfb5d34" containerName="maas-api"
Oct 30 21:26:46.121063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120912    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="787d1502-7802-4075-ab2b-152d7dfb5d34" containerName="maas-api"
Oct 30 21:26:46.121063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120925    6738 cpu_manager.go:414] "containerMap: removing container" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerName="maas-api"
Oct 30 21:26:46.121063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.120932    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerName="maas-api"
Oct 30 21:26:46.121269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.121076    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="097310d5-1f77-46d6-bd14-dbfafc5d4019" containerName="istio-proxy"
Oct 30 21:26:46.121269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.121093    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="787d1502-7802-4075-ab2b-152d7dfb5d34" containerName="maas-api"
Oct 30 21:26:46.121269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.121104    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbacc06c-ed14-44e0-a916-a40796ca7622" containerName="maas-api"
Oct 30 21:26:46.129251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.127813    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.143499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.138190    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:26:46.209568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.209517    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.210970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.210935    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.211328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.211312    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxqwf\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.211546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.211526    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.213843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.213817    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.213979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.213959    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.214068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.214010    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.214068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.214053    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.214183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.214086    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.291234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.291191    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:26:46.298581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.298555    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.307838    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-15e79c-serving-cert\""
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.314508    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315652    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315758    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315829    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315864    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315910    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.315955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.315938    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.316553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.316031    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.316553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.316057    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.316553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.316149    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxqwf\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.317258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.317230    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.319296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.318070    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.319296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.318124    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.319296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.318614    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.319296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.318852    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.322406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.322382    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.324959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.324933    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.371337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.371272    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxqwf\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.377939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.377911    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\") pod \"e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") " pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.417397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.417276    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.417397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.417383    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.418001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.417487    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.418001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.417534    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.458532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.458500    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.475844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.475800    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:46.519643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.519604    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.519846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.519711    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.519846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.519802    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.519956 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.519854    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.520757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.520726    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.520901 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:26:46.520729    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-serving-cert: secret "maas-api-e2e-shared-a-w3-15e79c-serving-cert" not found
Oct 30 21:26:46.520901 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:26:46.520839    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls podName:7c7b1113-9813-4257-bb65-3b1a03ce1455 nodeName:}" failed. No retries permitted until 2026-10-30 21:26:47.020818684 +0000 UTC m=+1463.976505904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls") pod "maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" (UID: "7c7b1113-9813-4257-bb65-3b1a03ce1455") : secret "maas-api-e2e-shared-a-w3-15e79c-serving-cert" not found
Oct 30 21:26:46.523894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.523861    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.551456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.549137    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:46.665908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.665831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:46.817534 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:26:46.817503    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62db81d7_7d94_4f2b_8755_e76ef9f5541f.slice/crio-967dd27adee76aae7e62470f1f84de0bc6a3d093c0ecb7d30160a5e0a558342b WatchSource:0}: Error finding container 967dd27adee76aae7e62470f1f84de0bc6a3d093c0ecb7d30160a5e0a558342b: Status 404 returned error can't find the container with id 967dd27adee76aae7e62470f1f84de0bc6a3d093c0ecb7d30160a5e0a558342b
Oct 30 21:26:46.818538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.818507    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:26:46.820186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.820151    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:46.820265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.820223    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:46.820265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.820259    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:26:46.873287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:46.873240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:47.044467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.031390    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:47.044467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.037044    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:47.274732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.274656    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:47.614861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.614833    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:26:47.615545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.615519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:47.653447 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.653405    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" event={"ID":"7c7b1113-9813-4257-bb65-3b1a03ce1455","Type":"ContainerStarted","Data":"e760cdcbd4c94c2293fa20e03d0c7a6a8b66de00a78bfca9c32925b57f799511"}
Oct 30 21:26:47.655322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.655299    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" event={"ID":"62db81d7-7d94-4f2b-8755-e76ef9f5541f","Type":"ContainerStarted","Data":"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"}
Oct 30 21:26:47.655486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.655341    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" event={"ID":"62db81d7-7d94-4f2b-8755-e76ef9f5541f","Type":"ContainerStarted","Data":"967dd27adee76aae7e62470f1f84de0bc6a3d093c0ecb7d30160a5e0a558342b"}
Oct 30 21:26:47.689979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.686416    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podStartSLOduration=1.686395111 podStartE2EDuration="1.686395111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:47.685707053 +0000 UTC m=+1464.641394281" watchObservedRunningTime="2026-10-30 21:26:47.686395111 +0000 UTC m=+1464.642082341"
Oct 30 21:26:47.748340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:47.748305    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:48.477077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:48.477046    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:48.478890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:48.478865    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:48.478988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:48.478911    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:48.654264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:48.654230    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:48.669481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:48.669415    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" event={"ID":"7c7b1113-9813-4257-bb65-3b1a03ce1455","Type":"ContainerStarted","Data":"0e4ec613fe4c49731fc12029cb1c37e6ab77dcbf7f7815e557e7f67feede8afc"}
Oct 30 21:26:49.477079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.477045    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:49.477253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.477113    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:49.584305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.584269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:49.707462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.706748    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:49.732518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.731916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:49.851473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:49.851406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:50.477424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:50.477275    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:50.477424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:50.477341    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:50.751707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:50.751629    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:51.477217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:51.477188    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:51.477388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:51.477235    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:51.761933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:51.761823    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:52.477200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:52.477164    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:52.477349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:52.477225    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:52.696821 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:26:52.696779    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:26:53.083250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:53.083222    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:53.476528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:53.476485    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused" start-of-body=
Oct 30 21:26:53.476736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:53.476561    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.85:15021/healthz/ready\": dial tcp 10.134.0.85:15021: connect: connection refused"
Oct 30 21:26:54.534950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:54.534888    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:55.001459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.001406    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:55.002975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.002944    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:55.003806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.003775    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:26:55.026138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.026094    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" podStartSLOduration=8.580681549 podStartE2EDuration="9.026079337s" totalImagesPullingTime="445.397788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:26:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:26:48.712081838 +0000 UTC m=+1465.667769067" watchObservedRunningTime="2026-10-30 21:26:55.026079337 +0000 UTC m=+1471.981766610"
Oct 30 21:26:55.494032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.493994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:55.734971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:55.734943    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:26:56.262745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:56.262714    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:57.523612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:57.523581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:57.767143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:57.767107    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.218733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:58.218693    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.379460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:58.379287    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:58.516711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:58.516614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:26:59.442083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:26:59.442023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:00.287294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:00.287256    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.125244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:01.125210    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.380006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:01.379935    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.596644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:01.596615    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:01.852737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:01.852655    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.191122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.191091    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.315957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.315895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.723287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.723252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.871358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.871288    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:02.973664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.973585    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:27:02.979550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.979530    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:02.987025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.986983    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-6ed261\""
Oct 30 21:27:02.988848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:02.988825    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-6ed261-dockercfg-pn4sx\""
Oct 30 21:27:03.003423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.003400    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:27:03.018264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.018229    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:27:03.032156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.032129    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.036583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.036563    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.046195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.045910    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:27:03.061904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.059480    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.061904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.059653    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlg9g\" (UniqueName: \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.061904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.059746    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.061904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.059891    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9bm\" (UniqueName: \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.225537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.224073    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlg9g\" (UniqueName: \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.225537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.224185    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.225537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.224384    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9bm\" (UniqueName: \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.225537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.224534    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.253544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.227972    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.255408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.255376    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.286492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.277743    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9bm\" (UniqueName: \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\") pod \"payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.308178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.308144    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlg9g\" (UniqueName: \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\") pod \"payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.311766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.310807    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.382766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.382522    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.663592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.663566    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:27:03.665869 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:03.665842    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c0c3504_66bc_4976_a053_61ec06d5a548.slice/crio-b971235506826ee12f4847435ad6951c88a834173ad2d9ed1297cb4b91b76af7 WatchSource:0}: Error finding container b971235506826ee12f4847435ad6951c88a834173ad2d9ed1297cb4b91b76af7: Status 404 returned error can't find the container with id b971235506826ee12f4847435ad6951c88a834173ad2d9ed1297cb4b91b76af7
Oct 30 21:27:03.770139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.770083    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:27:03.774587 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:03.774564    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be9f280_09d3_4aea_a658_ba19946d72d1.slice/crio-fdf484a9671d77913ac766af94af01e2b44bb32129ed675de0fb354a727f3db1 WatchSource:0}: Error finding container fdf484a9671d77913ac766af94af01e2b44bb32129ed675de0fb354a727f3db1: Status 404 returned error can't find the container with id fdf484a9671d77913ac766af94af01e2b44bb32129ed675de0fb354a727f3db1
Oct 30 21:27:03.841858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.841833    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:03.919951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.919000    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" event={"ID":"2be9f280-09d3-4aea-a658-ba19946d72d1","Type":"ContainerStarted","Data":"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"}
Oct 30 21:27:03.919951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.919068    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" event={"ID":"2be9f280-09d3-4aea-a658-ba19946d72d1","Type":"ContainerStarted","Data":"fdf484a9671d77913ac766af94af01e2b44bb32129ed675de0fb354a727f3db1"}
Oct 30 21:27:03.919951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.919098    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:03.922687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.922656    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" event={"ID":"6c0c3504-66bc-4976-a053-61ec06d5a548","Type":"ContainerStarted","Data":"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"}
Oct 30 21:27:03.922852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.922738    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" event={"ID":"6c0c3504-66bc-4976-a053-61ec06d5a548","Type":"ContainerStarted","Data":"b971235506826ee12f4847435ad6951c88a834173ad2d9ed1297cb4b91b76af7"}
Oct 30 21:27:03.923851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.923831    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:03.950297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.950242    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" podStartSLOduration=1.9502271979999999 podStartE2EDuration="1.950227198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:03.946792287 +0000 UTC m=+1480.902479515" watchObservedRunningTime="2026-10-30 21:27:03.950227198 +0000 UTC m=+1480.905914428"
Oct 30 21:27:03.968694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:03.968645    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" podStartSLOduration=1.968631783 podStartE2EDuration="1.968631783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:03.964755141 +0000 UTC m=+1480.920442370" watchObservedRunningTime="2026-10-30 21:27:03.968631783 +0000 UTC m=+1480.924319011"
Oct 30 21:27:04.328355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:04.328322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:04.894219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:04.894184    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:05.484828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:05.484775    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:05.981121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:05.981076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:06.318509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:06.317924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:06.653882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:06.653850    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:06.876558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:06.875249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:07.048521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:07.042269    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:07.798291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:07.798248    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:08.003503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:08.003470    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:08.269465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:08.257277    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.457946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.457888    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.584276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.584245    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:27:09.589322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.589299    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.605106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.605081    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-a9230f-dockercfg-t24jc\""
Oct 30 21:27:09.610665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.610635    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.610899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.610861    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfq6x\" (UniqueName: \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.611471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.611412    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-a9230f\""
Oct 30 21:27:09.614753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.614728    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:27:09.713619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.713542    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfq6x\" (UniqueName: \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.713816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.713796    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.714743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.714718    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.715736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.715700    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:27:09.721406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.721381    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:09.743375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.743338    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:27:09.746527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.746133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:09.773183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.773139    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfq6x\" (UniqueName: \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\") pod \"payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:09.911840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:09.911792    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:10.016517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.013946    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7n96\" (UniqueName: \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.016517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.014137    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.164463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.163774    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.164463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.164092    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7n96\" (UniqueName: \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.168453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.165522    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.174834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.174785    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7n96\" (UniqueName: \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\") pod \"payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.367150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.366707    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:10.658167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.624269    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:27:10.698478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:10.689982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:10.853460 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:10.850807    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f5e1c6_18ae_4d6a_aedb_1bbe002a97de.slice/crio-9f62ed21f3f8bd934edc8762d093021ded282272cb250f8e6cf3b13eed85e3e8 WatchSource:0}: Error finding container 9f62ed21f3f8bd934edc8762d093021ded282272cb250f8e6cf3b13eed85e3e8: Status 404 returned error can't find the container with id 9f62ed21f3f8bd934edc8762d093021ded282272cb250f8e6cf3b13eed85e3e8
Oct 30 21:27:11.018991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:11.018628    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:11.150451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:11.129535    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:27:11.270367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:11.270330    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" event={"ID":"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de","Type":"ContainerStarted","Data":"9f62ed21f3f8bd934edc8762d093021ded282272cb250f8e6cf3b13eed85e3e8"}
Oct 30 21:27:12.282474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.282412    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" event={"ID":"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de","Type":"ContainerStarted","Data":"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"}
Oct 30 21:27:12.287023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.286999    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:12.289147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.289116    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" event={"ID":"f3126c2f-abf7-414c-904a-cc39332c3e84","Type":"ContainerStarted","Data":"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"}
Oct 30 21:27:12.289362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.289336    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" event={"ID":"f3126c2f-abf7-414c-904a-cc39332c3e84","Type":"ContainerStarted","Data":"65a14ff9cdafba3ea62d8e906fd733b2c7ce75cddb33e6692bc6249754ae1f4f"}
Oct 30 21:27:12.290419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.290399    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:12.414548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.414476    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" podStartSLOduration=3.414456058 podStartE2EDuration="3.414456058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:12.411213053 +0000 UTC m=+1489.366900284" watchObservedRunningTime="2026-10-30 21:27:12.414456058 +0000 UTC m=+1489.370143287"
Oct 30 21:27:12.417605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.417560    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" podStartSLOduration=3.417544713 podStartE2EDuration="3.417544713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:12.335174715 +0000 UTC m=+1489.290861949" watchObservedRunningTime="2026-10-30 21:27:12.417544713 +0000 UTC m=+1489.373231942"
Oct 30 21:27:12.604469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.604362    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:12.946338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:12.946305    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:13.152230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:13.152197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:13.825119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:13.825084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:14.942678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:14.942646    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:27:15.000328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.000291    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:15.922661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.922629    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:27:15.927114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.927094    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:15.934317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.934295    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-6ed261-serving-cert\""
Oct 30 21:27:15.944595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.944574    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:27:15.945057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:15.945036    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:27:16.002512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.002482    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.002693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.002653    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86h4q\" (UniqueName: \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.002785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.002769    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.002871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.002854    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.094663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.094576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.105193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.104516    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.105193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.104601    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.105193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.104738    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.105193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.104821    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86h4q\" (UniqueName: \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.108063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.107737    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.124080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.124015    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.124985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.124936    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.128449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.128415    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86h4q\" (UniqueName: \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\") pod \"maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.242003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.241931    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:16.302035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.302002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:16.553662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.553633    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:27:16.555071 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:16.555037    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95571c93_227f_4acd_a730_921faa0c2cab.slice/crio-c34883cdadd2b2bd276a99ddc1dd182cf0e89c0114354260896cc77487ee3ab6 WatchSource:0}: Error finding container c34883cdadd2b2bd276a99ddc1dd182cf0e89c0114354260896cc77487ee3ab6: Status 404 returned error can't find the container with id c34883cdadd2b2bd276a99ddc1dd182cf0e89c0114354260896cc77487ee3ab6
Oct 30 21:27:16.557557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:16.557423    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:27:17.360415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:17.360337    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" event={"ID":"95571c93-227f-4acd-a730-921faa0c2cab","Type":"ContainerStarted","Data":"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"}
Oct 30 21:27:17.360415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:17.360393    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" event={"ID":"95571c93-227f-4acd-a730-921faa0c2cab","Type":"ContainerStarted","Data":"c34883cdadd2b2bd276a99ddc1dd182cf0e89c0114354260896cc77487ee3ab6"}
Oct 30 21:27:17.360916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:17.360483    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:17.408987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:17.408926    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" podStartSLOduration=1.9813431559999999 podStartE2EDuration="2.408905718s" totalImagesPullingTime="427.562562ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:17.405341098 +0000 UTC m=+1494.361028330" watchObservedRunningTime="2026-10-30 21:27:17.408905718 +0000 UTC m=+1494.364592946"
Oct 30 21:27:17.475129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:17.474982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:18.582610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:18.582573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:18.844884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:18.844801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:19.497151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:19.497084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:20.290675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:20.290649    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.098231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:21.098200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.867119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:21.867095    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:21.996093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:21.996066    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.567982 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:27:22.567938    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:27:22.697826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.697794    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:27:22.703918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.703893    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.711803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.711777    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-a9230f-serving-cert\""
Oct 30 21:27:22.717206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.717176    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:27:22.786873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.786834    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:22.818475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.818379    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.818618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.818518    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.818681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.818652    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.818810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.818794    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndb6d\" (UniqueName: \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.920110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.920061    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.920867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.920831    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndb6d\" (UniqueName: \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.921183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.921158    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.921374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.921354    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.922511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.922404    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.925213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.925159    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.926694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.926670    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.941688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.941662    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndb6d\" (UniqueName: \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\") pod \"maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:22.985773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:22.985745    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:23.030859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:23.030825    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:23.340513 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:23.338085    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670ad8df_c084_4e8a_bb8f_6de1017cb331.slice/crio-1eed9f3e7eccff6d97fbd83018c205e7bc66b69927bf57d490968d0a944640ff WatchSource:0}: Error finding container 1eed9f3e7eccff6d97fbd83018c205e7bc66b69927bf57d490968d0a944640ff: Status 404 returned error can't find the container with id 1eed9f3e7eccff6d97fbd83018c205e7bc66b69927bf57d490968d0a944640ff
Oct 30 21:27:23.343816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:23.343757    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:27:23.389459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:23.386998    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:27:23.497461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:23.490197    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" event={"ID":"670ad8df-c084-4e8a-bb8f-6de1017cb331","Type":"ContainerStarted","Data":"1eed9f3e7eccff6d97fbd83018c205e7bc66b69927bf57d490968d0a944640ff"}
Oct 30 21:27:23.870498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:23.870109    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:24.336611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.336580    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:27:24.337272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.337249    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:27:24.498473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.498415    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:27:24.498733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.498700    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy" containerID="cri-o://93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069" gracePeriod=30
Oct 30 21:27:24.500248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.500172    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" event={"ID":"670ad8df-c084-4e8a-bb8f-6de1017cb331","Type":"ContainerStarted","Data":"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"}
Oct 30 21:27:24.500414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.500255    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:24.566005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.565759    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" podStartSLOduration=2.127373541 podStartE2EDuration="2.565737641s" totalImagesPullingTime="438.3641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:24.565094351 +0000 UTC m=+1501.520781603" watchObservedRunningTime="2026-10-30 21:27:24.565737641 +0000 UTC m=+1501.521424905"
Oct 30 21:27:24.805669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:24.805630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:25.532469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:25.532408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:25.657027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:25.656335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:26.559646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:26.559613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:27.655462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:27.655020    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:27.927759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:27.927682    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:28.758389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:28.757717    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.657051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.656960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:29.819849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.819824    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:27:29.882170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882140    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.882345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882210    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.882345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882312    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.882422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882406    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.882497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882481    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882487    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882615    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882676    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882818    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882851    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882855    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.883176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.882930    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\") pod \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\" (UID: \"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074\") "
Oct 30 21:27:29.883660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.883289    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:29.883741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.883667    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:29.884022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.883997    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.884120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884026    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.884120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884042    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.884120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884055    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.884120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884068    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.884940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884916    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:27:29.884940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.884930    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:29.885173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.885156    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "kube-api-access-rjx9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:27:29.885238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.885222    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo" pod "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" (UID: "0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:27:29.986105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.986068    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.986105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.986108    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.986314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.986126    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjx9c\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-kube-api-access-rjx9c\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:29.986314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:29.986142    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:27:30.513142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.513102    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:27:30.562094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.562061    6738 generic.go:356] "Generic (PLEG): container finished" podID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerID="93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069" exitCode=0
Oct 30 21:27:30.562264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.562117    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"
Oct 30 21:27:30.562264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.562121    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" event={"ID":"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074","Type":"ContainerDied","Data":"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"}
Oct 30 21:27:30.562264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.562185    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt" event={"ID":"0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074","Type":"ContainerDied","Data":"9e939ef5bb5fb74aa39250b290bb98dee2f52d3e2b618352f797fe92fe970ab8"}
Oct 30 21:27:30.562264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.562216    6738 scope.go:122] "RemoveContainer" containerID="93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"
Oct 30 21:27:30.584304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.583450    6738 scope.go:122] "RemoveContainer" containerID="93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"
Oct 30 21:27:30.585894 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:27:30.585793    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069\": container with ID starting with 93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069 not found: ID does not exist" containerID="93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"
Oct 30 21:27:30.585894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.585832    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069"} err="failed to get container status \"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069\": rpc error: code = NotFound desc = could not find container \"93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069\": container with ID starting with 93735cc4dd7bbd008ac9fbcb46a233d71af56163791431a7b5c270cba59bd069 not found: ID does not exist"
Oct 30 21:27:30.610689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.610515    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:27:30.620847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.620795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:30.621552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:30.621524    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-ck7jt"]
Oct 30 21:27:31.165810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:31.165783    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" path="/var/lib/kubelet/pods/0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074/volumes"
Oct 30 21:27:31.791932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:31.790302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:32.941841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:32.941803    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:33.878153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:33.878120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:35.257919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:35.257875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:36.676863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:36.676831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:37.049931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:37.049856    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:37.935902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:37.935857    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:39.054761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:39.054397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:40.231130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:40.231098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:41.055356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:41.055326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:42.012065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.012037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:42.014794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.014766    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:27:42.015766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.015745    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy"
Oct 30 21:27:42.015766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.015766    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy"
Oct 30 21:27:42.015943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.015931    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bfc79d6-0e95-42ba-a18d-9d3b2e8b8074" containerName="istio-proxy"
Oct 30 21:27:42.042926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.042887    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:27:42.043083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.042996    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.053983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.053949    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-06bb0f\"/\"kube-root-ca.crt\""
Oct 30 21:27:42.054945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.054920    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-06bb0f\"/\"openshift-service-ca.crt\""
Oct 30 21:27:42.055311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.055269    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-06bb0f\"/\"default-dockercfg-glvp5\""
Oct 30 21:27:42.060277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.060256    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-06bb0f\"/\"auth-test-model-06bb0f-kserve-self-signed-certs\""
Oct 30 21:27:42.135816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.135778    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.135994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.135843    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmqcw\" (UniqueName: \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.136130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.136101    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.136226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.136209    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.136308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.136289    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.136384 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.136354    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.253466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.252480    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:42.261427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.261383    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.261764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.261741    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.261893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.261880    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.262024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.262011    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.262264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.262211    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.262403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.262388    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmqcw\" (UniqueName: \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.263336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.263313    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.263755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.263733    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.269514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.266849    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.277491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.275213    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.277491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.275520    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.302465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.294548    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmqcw\" (UniqueName: \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\") pod \"auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") " pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.369795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.369158    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:42.704843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:42.704816    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:27:42.705677 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:27:42.705650    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07591128_d54d_45ff_a62d_befd18719dcc.slice/crio-f55d5892783aa875b73849577475474154f4868bd033bcbb3cbc5ce36f76fcaf WatchSource:0}: Error finding container f55d5892783aa875b73849577475474154f4868bd033bcbb3cbc5ce36f76fcaf: Status 404 returned error can't find the container with id f55d5892783aa875b73849577475474154f4868bd033bcbb3cbc5ce36f76fcaf
Oct 30 21:27:43.198731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:43.198695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:43.712741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:43.712701    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" event={"ID":"07591128-d54d-45ff-a62d-befd18719dcc","Type":"ContainerStarted","Data":"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"}
Oct 30 21:27:43.712911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:43.712756    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" event={"ID":"07591128-d54d-45ff-a62d-befd18719dcc","Type":"ContainerStarted","Data":"f55d5892783aa875b73849577475474154f4868bd033bcbb3cbc5ce36f76fcaf"}
Oct 30 21:27:43.790462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:43.790386    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" podStartSLOduration=2.790372035 podStartE2EDuration="2.790372035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:43.786704531 +0000 UTC m=+1520.742391760" watchObservedRunningTime="2026-10-30 21:27:43.790372035 +0000 UTC m=+1520.746059262"
Oct 30 21:27:44.370385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:44.370350    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:44.621333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:44.621254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:45.721905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:45.721872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:46.304063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:46.304033    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:46.791930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:46.791896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.135493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.135455    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.564511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.558249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.817550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.815597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:47.937785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.937747    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:27:47.942501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.942359    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:47.975413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.975374    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:27:47.978334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:47.978305    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-a9230f-openshift-default-dockercfg-7qqkw\""
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.073823    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.073876    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6shh9\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.073937    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.073978    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.074045    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.074137    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.074162    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.074241    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.076518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.074269    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.177801    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.177887    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6shh9\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.177922    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.177952    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178043    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178118    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178154    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178222    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178263    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.179469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.178816    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.181161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.180483    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.181161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.180757    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.187036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.187007    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.187920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.187894    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.194110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.194082    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.197476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.195083    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.197476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.195577    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.197476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.195691    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6shh9\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.275605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.275574    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:48.508513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.508472    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:48.594806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.594753    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:27:48.599779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.598743    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:48.599779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.598836    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:48.599779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.598886    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:27:48.836461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.833821    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" event={"ID":"1f4d5198-328b-4c04-bce2-7202b2d7aa7a","Type":"ContainerStarted","Data":"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"}
Oct 30 21:27:48.836461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.833880    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" event={"ID":"1f4d5198-328b-4c04-bce2-7202b2d7aa7a","Type":"ContainerStarted","Data":"0c3dfe0b3c88843ad3dc55fc54f5b4b31acc99c236eeebda1d9ed72be688e8c6"}
Oct 30 21:27:48.894637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:48.894576    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podStartSLOduration=1.8945605859999999 podStartE2EDuration="1.894560586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:27:48.891991427 +0000 UTC m=+1525.847678661" watchObservedRunningTime="2026-10-30 21:27:48.894560586 +0000 UTC m=+1525.850247819"
Oct 30 21:27:49.276716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:49.276617    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:49.280292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:49.280258    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:49.280496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:49.280318    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:50.127195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:50.126303    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:50.276595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:50.276562    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:50.276765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:50.276613    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:51.276370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:51.276341    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:51.276808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:51.276396    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:51.410460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:51.406203    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:51.714460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:51.706949    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:52.277102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.277068    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:52.277657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.277121    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:52.369830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.369794    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:52.387196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.387172    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:52.566723 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:27:52.566615    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:27:52.882023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.881992    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:27:52.954044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:52.954011    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:53.280524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:53.277823    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:53.280524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:53.277884    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:53.308643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:53.303928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:53.517462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:53.510044    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:54.276140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:54.276105    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Oct 30 21:27:54.276457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:54.276402    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Oct 30 21:27:55.210388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:55.210352    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:55.282558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:55.282532    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:55.282849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:55.282823    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:55.283589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:55.283574    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:27:56.781633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:56.781597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:58.109396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:58.109325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:58.732482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:58.732448    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.615814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.615781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.627509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.627472    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:27:59.634131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.634108    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.649547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.649521    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-726a18\"/\"default-dockercfg-xchgp\""
Oct 30 21:27:59.649772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.649751    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w4-726a18\"/\"auth-test-model-726a18-kserve-self-signed-certs\""
Oct 30 21:27:59.651810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.651788    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:27:59.657070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.657051    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-726a18\"/\"openshift-service-ca.crt\""
Oct 30 21:27:59.657228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.657115    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w4-726a18\"/\"kube-root-ca.crt\""
Oct 30 21:27:59.766072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766033    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.766233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766182    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.766281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766266    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.766373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766352    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.766464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766413    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.766562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.766546    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.870529    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.870595    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.870649    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.870738    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.870985    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.871509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.871082    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.875461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.873128    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.881674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.881507    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.882010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.881969    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:27:59.882455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.882408    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.882709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.882675    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.891079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.891030    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.924316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.924277    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\") pod \"auth-test-model-726a18-kserve-84f789cbc7-bpjsp\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") " pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:27:59.983158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:27:59.983098    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:28:00.334998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:00.334968    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:28:00.337806 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:28:00.337775    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ec62f94_4382_4259_ac0f_2d961c8f67af.slice/crio-30852467efcaf8485dbaf030856ff71983a852b5980e640c6d68bd8d7edce91c WatchSource:0}: Error finding container 30852467efcaf8485dbaf030856ff71983a852b5980e640c6d68bd8d7edce91c: Status 404 returned error can't find the container with id 30852467efcaf8485dbaf030856ff71983a852b5980e640c6d68bd8d7edce91c
Oct 30 21:28:00.906154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:00.906108    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.001046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:01.001007    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" event={"ID":"1ec62f94-4382-4259-ac0f-2d961c8f67af","Type":"ContainerStarted","Data":"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"}
Oct 30 21:28:01.001216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:01.001064    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" event={"ID":"1ec62f94-4382-4259-ac0f-2d961c8f67af","Type":"ContainerStarted","Data":"30852467efcaf8485dbaf030856ff71983a852b5980e640c6d68bd8d7edce91c"}
Oct 30 21:28:01.035654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:01.035594    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" podStartSLOduration=2.035574541 podStartE2EDuration="2.035574541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:01.033463772 +0000 UTC m=+1537.989151004" watchObservedRunningTime="2026-10-30 21:28:01.035574541 +0000 UTC m=+1537.991261771"
Oct 30 21:28:01.885266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:01.885232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:01.988825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:01.988794    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:28:03.002913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:03.002870    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:03.817348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:03.817313    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:04.165418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:04.165370    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:04.388090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:04.387970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:05.672964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:05.672864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:06.791528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:06.791486    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:07.750759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:07.750721    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:08.906233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:08.906186    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:09.900099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:09.900061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:09.988565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:09.988532    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:28:10.006309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.006280    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:28:10.086485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.086431    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:28:10.841698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.841658    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:28:10.846974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.846949    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:10.853874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.853837    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w1-27804d-serving-cert\""
Oct 30 21:28:10.865490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.865466    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:28:10.933087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.932965    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:10.964040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.963752    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qgsm\" (UniqueName: \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:10.964040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.963942    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:10.964342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.964046    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:10.964342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:10.964126    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.065682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.065649    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.066101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.065794    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qgsm\" (UniqueName: \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.066101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.065899    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.066101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.065974    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.066701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.066603    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.068310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.068286    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.068530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.068509    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.080303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.080274    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qgsm\" (UniqueName: \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\") pod \"maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.143394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.143363    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:11.164038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.164013    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:11.464191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.464158    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:28:11.466910 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:28:11.466882    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaa40822_2e8f_4f91_8a17_3febe3a476fc.slice/crio-52575a4986e4d30d185bc4a5aac4be03848df97b00e83ff83b4f9af2fd50b2dc WatchSource:0}: Error finding container 52575a4986e4d30d185bc4a5aac4be03848df97b00e83ff83b4f9af2fd50b2dc: Status 404 returned error can't find the container with id 52575a4986e4d30d185bc4a5aac4be03848df97b00e83ff83b4f9af2fd50b2dc
Oct 30 21:28:11.664087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:11.664029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:12.085108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:12.085074    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" event={"ID":"baa40822-2e8f-4f91-8a17-3febe3a476fc","Type":"ContainerStarted","Data":"52575a4986e4d30d185bc4a5aac4be03848df97b00e83ff83b4f9af2fd50b2dc"}
Oct 30 21:28:12.477629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:12.477541    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:13.093686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:13.093644    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" event={"ID":"baa40822-2e8f-4f91-8a17-3febe3a476fc","Type":"ContainerStarted","Data":"d1fa3ab5877ed28630fdb62706ce90e343d965e701256d3a160242a739558e4b"}
Oct 30 21:28:13.094675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:13.094649    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:13.940285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:13.940254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:15.035456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:15.035410    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:16.284468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:16.284414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:17.739771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:17.739732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:18.049479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.049355    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" podStartSLOduration=7.519834581 podStartE2EDuration="8.049335624s" totalImagesPullingTime="529.501043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:13.122537511 +0000 UTC m=+1550.078224738" watchObservedRunningTime="2026-10-30 21:28:18.049335624 +0000 UTC m=+1555.005022853"
Oct 30 21:28:18.050818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.050792    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:28:18.058779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.058744    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.094100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.094070    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:28:18.196318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196283    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196451    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196484    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196575    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfmss\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196605    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196770    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196813    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196943    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.196996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.196968    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.297989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.297958    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.297989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.297992    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298062    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298077    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298100    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298172    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298188    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298238    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfmss\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298279    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298527    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298544    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298632    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298697    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.298987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.298960    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.300937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.300881    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.301221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.301198    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.321273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.321246    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.321415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.321396    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfmss\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.379241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.379211    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:18.712589 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:28:18.712552    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff9b43a_2ac7_4a33_8bc4_ef6e4d4fafe4.slice/crio-e26ec5d6f38aa2aa95f910e71590c333d805f81fa99e523e225cb7c100b9c535 WatchSource:0}: Error finding container e26ec5d6f38aa2aa95f910e71590c333d805f81fa99e523e225cb7c100b9c535: Status 404 returned error can't find the container with id e26ec5d6f38aa2aa95f910e71590c333d805f81fa99e523e225cb7c100b9c535
Oct 30 21:28:18.714009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.713979    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:28:18.715641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.715598    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:18.715753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.715715    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:18.715793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.715762    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:18.925474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:18.925375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:19.144671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.144608    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" event={"ID":"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4","Type":"ContainerStarted","Data":"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"}
Oct 30 21:28:19.144846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.144699    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" event={"ID":"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4","Type":"ContainerStarted","Data":"e26ec5d6f38aa2aa95f910e71590c333d805f81fa99e523e225cb7c100b9c535"}
Oct 30 21:28:19.151328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.151299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:19.176180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.176129    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podStartSLOduration=2.176115964 podStartE2EDuration="2.176115964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:19.17168149 +0000 UTC m=+1556.127368709" watchObservedRunningTime="2026-10-30 21:28:19.176115964 +0000 UTC m=+1556.131803191"
Oct 30 21:28:19.380200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.380156    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:19.381718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.381692    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:19.381857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:19.381782    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:20.123200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:20.123078    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:28:20.380535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:20.380411    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:20.380715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:20.380528    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:20.444157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:20.444097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:21.380443 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:21.380411    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:21.380878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:21.380484    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:21.641818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:21.641723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:22.380154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:22.380123    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:22.380316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:22.380178    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:22.552191 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:28:22.552137    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:28:22.814295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:22.814200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:23.380593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:23.380557    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:23.380763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:23.380612    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:24.195702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:24.195657    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:24.379783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:24.379753    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused" start-of-body=
Oct 30 21:28:24.379954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:24.379812    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.97:15021/healthz/ready\": dial tcp 10.134.0.97:15021: connect: connection refused"
Oct 30 21:28:25.359111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:25.359069    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:25.384785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:25.384760    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:25.385178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:25.385159    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:25.385805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:25.385786    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:28:27.151526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.151491    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:28:27.158807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.158776    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.175770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.175737    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a9230f\"/\"openshift-service-ca.crt\""
Oct 30 21:28:27.175932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.175887    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-a9230f\"/\"kube-root-ca.crt\""
Oct 30 21:28:27.186451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.184063    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:28:27.218642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.218549    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a9230f\"/\"default-dockercfg-pn2fx\""
Oct 30 21:28:27.220337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.220312    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-a9230f\"/\"auth-test-model-a9230f-kserve-self-signed-certs\""
Oct 30 21:28:27.221195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.221167    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.221394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.221376    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.221557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.221539    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvhj2\" (UniqueName: \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.221665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.221652    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.221962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.221945    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.222368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.222347    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.265981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.265942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:27.323951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.323907    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.323951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.323957    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.324243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.323989    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvhj2\" (UniqueName: \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.324243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.324015    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.324243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.324170    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.324409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.324295    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.324519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.324473    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.325491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.324813    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.326142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.326113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.327713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.327689    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.329188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.329162    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.346427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.346399    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvhj2\" (UniqueName: \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\") pod \"auth-test-model-a9230f-kserve-79f7d45945-jrj9c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") " pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.489583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.488118    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:27.588456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.569926    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:27.931231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:27.931199    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:28:27.933306 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:28:27.933282    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a858696_3ba6_47ba_a6fe_7bb9ae2ba58c.slice/crio-7d9dffc15ec595359007d33cc032dd206992f0867eacdd6dda1a62cefa8de4a4 WatchSource:0}: Error finding container 7d9dffc15ec595359007d33cc032dd206992f0867eacdd6dda1a62cefa8de4a4: Status 404 returned error can't find the container with id 7d9dffc15ec595359007d33cc032dd206992f0867eacdd6dda1a62cefa8de4a4
Oct 30 21:28:28.221214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:28.216695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:28.241080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:28.241046    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" event={"ID":"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c","Type":"ContainerStarted","Data":"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"}
Oct 30 21:28:28.241252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:28.241090    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" event={"ID":"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c","Type":"ContainerStarted","Data":"7d9dffc15ec595359007d33cc032dd206992f0867eacdd6dda1a62cefa8de4a4"}
Oct 30 21:28:28.286589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:28.286547    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" podStartSLOduration=1.286534381 podStartE2EDuration="1.286534381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:28.282913733 +0000 UTC m=+1565.238600961" watchObservedRunningTime="2026-10-30 21:28:28.286534381 +0000 UTC m=+1565.242221608"
Oct 30 21:28:28.488757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:28.488654    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:29.218635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:29.218540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:29.451342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:29.451309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:30.548295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:30.548258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:31.788048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:31.787982    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:33.042890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.042860    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:28:33.047805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.047780    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.093478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.093399    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:28:33.159839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.159803    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.159908    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74k2s\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.159988    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160054    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160096    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160200    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160297    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160378    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.160583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.160499    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.261776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.261749    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.261937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.261839    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262018    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262105    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262149    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262245    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74k2s\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262333    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262365    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262400    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262464    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.262917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262891    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.263086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262945    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.263086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.262960    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.263229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.263188    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.264646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.264615    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.264812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.264788    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.278347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.278324    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.292260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.292232    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74k2s\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\") pod \"e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") " pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.374027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.373730    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:33.469462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:33.469210    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:34.093156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:34.092793    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:28:34.115737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:34.115185    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:34.115737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:34.115282    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:34.115737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:34.115322    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:28:34.288113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:34.288080    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" event={"ID":"8808f347-188b-4cf9-8d64-7946fdd523f1","Type":"ContainerStarted","Data":"d8fc08abb83a92ec1d955c2d71fa389ba1e6fb601f113d333465e1975dcb6c66"}
Oct 30 21:28:35.308471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.308146    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" event={"ID":"8808f347-188b-4cf9-8d64-7946fdd523f1","Type":"ContainerStarted","Data":"c687d922dcbcf69a1c8b9039aa2d00523216060ce8b6b227555369a77935dac1"}
Oct 30 21:28:35.370403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.370262    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podStartSLOduration=2.370246876 podStartE2EDuration="2.370246876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:35.367538167 +0000 UTC m=+1572.323225395" watchObservedRunningTime="2026-10-30 21:28:35.370246876 +0000 UTC m=+1572.325934206"
Oct 30 21:28:35.402459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.400176    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:35.402459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.400293    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:28:35.402459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.400337    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:28:35.611418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:35.608405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:36.401462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:36.397832    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:28:36.401462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:36.397888    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:28:36.932455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:36.932411    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:37.338620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.338523    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:37.412405 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.412375    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:28:37.412964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.412937    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:28:37.511461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.508675    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:37.532374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.532349    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:37.610308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:37.608794    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:38.315475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:38.314981    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:38.378536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:38.378508    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:28:38.418139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:38.417621    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:28:38.418139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:38.417670    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:28:39.405464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:39.398757    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:28:39.405464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:39.398815    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:28:40.033774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:40.033743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:40.410095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:40.410061    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:40.411131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:40.411103    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:40.411988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:40.411962    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:28:41.464929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:41.464903    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:42.649795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:42.649751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:43.993971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:43.993931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:45.026323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:45.026279    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:46.130502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:46.130467    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:47.168654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:47.168620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:48.769668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:48.769634    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:49.838058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:49.838006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:50.881150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:50.881106    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:52.393356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:52.393321    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:52.612560 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:28:52.612517    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:28:53.406825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.406791    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:28:53.420005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.419977    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.448746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.443404    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-d4a0a5\"/\"openshift-service-ca.crt\""
Oct 30 21:28:53.448746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.443556    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-d4a0a5\"/\"auth-test-model-d4a0a5-kserve-self-signed-certs\""
Oct 30 21:28:53.448746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.443624    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-d4a0a5\"/\"kube-root-ca.crt\""
Oct 30 21:28:53.448746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.443698    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-d4a0a5\"/\"default-dockercfg-m49xd\""
Oct 30 21:28:53.506300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506267    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wszwh\" (UniqueName: \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.506559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506322    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.506559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506490    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.506762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506739    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.506829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506814    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.506874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.506845    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616470    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616539    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616634    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wszwh\" (UniqueName: \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616676    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616743    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.616973    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.617451    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.618318    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.618566    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.626377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.625558    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.642860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.642751    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.655414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.655375    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wszwh\" (UniqueName: \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\") pod \"auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") " pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:53.678902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.678817    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:28:53.708182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.708140    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:53.775059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:53.774748    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:54.003103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.002882    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:54.107574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.107546    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:28:54.108409 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:28:54.108385    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f3926a8_795e_4558_b78e_8a27f6d66ec9.slice/crio-be96fa0bca7b5512f7259b10f4b84648f464b03fad0933e97216da7efdd8e67d WatchSource:0}: Error finding container be96fa0bca7b5512f7259b10f4b84648f464b03fad0933e97216da7efdd8e67d: Status 404 returned error can't find the container with id be96fa0bca7b5512f7259b10f4b84648f464b03fad0933e97216da7efdd8e67d
Oct 30 21:28:54.510262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.510219    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" event={"ID":"9f3926a8-795e-4558-b78e-8a27f6d66ec9","Type":"ContainerStarted","Data":"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"}
Oct 30 21:28:54.510262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.510264    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" event={"ID":"9f3926a8-795e-4558-b78e-8a27f6d66ec9","Type":"ContainerStarted","Data":"be96fa0bca7b5512f7259b10f4b84648f464b03fad0933e97216da7efdd8e67d"}
Oct 30 21:28:54.560244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.560184    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" podStartSLOduration=1.560165805 podStartE2EDuration="1.560165805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:28:54.555533827 +0000 UTC m=+1591.511221056" watchObservedRunningTime="2026-10-30 21:28:54.560165805 +0000 UTC m=+1591.515853033"
Oct 30 21:28:54.778660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:54.778583    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:28:55.502388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:55.502349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:56.691938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:56.691902    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:57.858361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:57.858330    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:58.945538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:58.944460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:28:59.971853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:28:59.971805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:01.285693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:01.285661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:01.633403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:01.633371    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:02.462483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:02.461040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:02.722386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:02.722284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:03.622235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:03.622163    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:03.778189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:03.778158    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:29:03.790681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:03.790655    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:29:04.628652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:04.628625    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:29:04.865377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:04.865332    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:05.965470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:05.965400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:06.992998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:06.992967    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.000480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:08.000422    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:29:08.237986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:08.237949    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.521844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:08.521319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:08.798243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:08.798167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:09.631461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:09.606385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:10.202122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:10.202092    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:10.927527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:10.927494    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:11.522775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:11.522749    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:12.736461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:12.736312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:13.751972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:13.751936    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:14.077092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:14.077019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:14.344423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:14.344355    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:15.374665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.374631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:15.523119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.523090    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:29:15.528323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.528302    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.546887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.546863    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-46c98c-dockercfg-tvh47\""
Oct 30 21:29:15.547897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.547872    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:29:15.610741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.610684    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.611107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.610753    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlrjx\" (UniqueName: \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.622457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.611203    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-46c98c\""
Oct 30 21:29:15.742025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.741945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:15.886923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.886886    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.887120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.887012    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlrjx\" (UniqueName: \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.887923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.887893    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:15.901565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:15.901535    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlrjx\" (UniqueName: \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\") pod \"payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:16.146642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.146612    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:16.458945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.458913    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:29:16.459799 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:29:16.459774    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4e7257_a7bb_403e_94a3_9fb9585f7a6d.slice/crio-33cb773f5e807bead00096912b1e8847ff87c0e7ffb510c289d2d34b5fb2dfbd WatchSource:0}: Error finding container 33cb773f5e807bead00096912b1e8847ff87c0e7ffb510c289d2d34b5fb2dfbd: Status 404 returned error can't find the container with id 33cb773f5e807bead00096912b1e8847ff87c0e7ffb510c289d2d34b5fb2dfbd
Oct 30 21:29:16.703924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.703856    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:16.839689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.839601    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" event={"ID":"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d","Type":"ContainerStarted","Data":"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"}
Oct 30 21:29:16.839689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.839683    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" event={"ID":"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d","Type":"ContainerStarted","Data":"33cb773f5e807bead00096912b1e8847ff87c0e7ffb510c289d2d34b5fb2dfbd"}
Oct 30 21:29:16.839922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.839749    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:16.885794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:16.885741    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" podStartSLOduration=1.885724089 podStartE2EDuration="1.885724089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:16.881122458 +0000 UTC m=+1613.836809685" watchObservedRunningTime="2026-10-30 21:29:16.885724089 +0000 UTC m=+1613.841411316"
Oct 30 21:29:18.410357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:18.410323    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:18.991739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:18.991691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:19.634709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:19.634680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:20.794281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:20.794245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:21.876491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:21.876430    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:22.123095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:22.123042    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:22.510622 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:29:22.510583    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:29:22.941788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:22.941751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:23.158560 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.158524    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerName="main" containerID="cri-o://34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506" gracePeriod=45
Oct 30 21:29:23.452549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.452527    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:29:23.525611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.525586    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\" (UniqueName: \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.525829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.525810    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.525897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.525882    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.525957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.525935    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.526025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526011    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.526082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526049    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\") pod \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\" (UID: \"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c\") "
Oct 30 21:29:23.526335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526223    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:23.526335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526201    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:23.526690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526431    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:23.526930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526904    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.527055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526936    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.527055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.526950    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.528154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.528127    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "kube-api-access-pvhj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:29:23.528519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.528500    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:29:23.528979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.528963    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs" pod "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" (UID: "9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:29:23.628717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.628688    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvhj2\" (UniqueName: \"kubernetes.io/projected/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-kube-api-access-pvhj2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.628717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.628713    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.628717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.628724    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:29:23.900220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.900182    6738 generic.go:356] "Generic (PLEG): container finished" podID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerID="34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506" exitCode=0
Oct 30 21:29:23.900423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.900356    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" event={"ID":"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c","Type":"ContainerDied","Data":"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"}
Oct 30 21:29:23.900423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.900395    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" event={"ID":"9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c","Type":"ContainerDied","Data":"7d9dffc15ec595359007d33cc032dd206992f0867eacdd6dda1a62cefa8de4a4"}
Oct 30 21:29:23.900423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.900418    6738 scope.go:122] "RemoveContainer" containerID="34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"
Oct 30 21:29:23.903479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.903429    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"
Oct 30 21:29:23.914720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.914699    6738 scope.go:122] "RemoveContainer" containerID="34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"
Oct 30 21:29:23.915013 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:29:23.914987    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506\": container with ID starting with 34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506 not found: ID does not exist" containerID="34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"
Oct 30 21:29:23.915115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.915022    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506"} err="failed to get container status \"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506\": rpc error: code = NotFound desc = could not find container \"34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506\": container with ID starting with 34ba4c109387ae926b96194410b7539b90226b16f008f7c2ad3ffecc0021f506 not found: ID does not exist"
Oct 30 21:29:23.947093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.947068    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:29:23.955338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:23.955316    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c"]
Oct 30 21:29:24.138953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:24.138921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:24.358415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:24.358325    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w3-a9230f/auth-test-model-a9230f-kserve-79f7d45945-jrj9c" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.98:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:29:25.165633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:25.165601    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" path="/var/lib/kubelet/pods/9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c/volumes"
Oct 30 21:29:25.940964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:25.940932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:26.134811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.134779    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:29:26.135550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.135530    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerName="main"
Oct 30 21:29:26.135550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.135549    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerName="main"
Oct 30 21:29:26.135815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.135800    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a858696-3ba6-47ba-a6fe-7bb9ae2ba58c" containerName="main"
Oct 30 21:29:26.141212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.141193    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.147798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.147778    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-27804d\"/\"default-dockercfg-xxctc\""
Oct 30 21:29:26.147944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.147810    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w1-27804d\"/\"auth-test-model-27804d-kserve-self-signed-certs\""
Oct 30 21:29:26.147944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.147810    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-27804d\"/\"kube-root-ca.crt\""
Oct 30 21:29:26.175457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.175407    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:29:26.249718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.249655    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w1-27804d\"/\"openshift-service-ca.crt\""
Oct 30 21:29:26.280797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.280763    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.280954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.280804    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.280954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.280912    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.281026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.281008    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.281100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.281082    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv5s7\" (UniqueName: \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.281220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.281205    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.346300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.346254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:26.382989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.382958    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.383528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.383496    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.383918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.383900    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.384279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.384259    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.393473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.384590    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv5s7\" (UniqueName: \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.393748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.391873    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.393839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.392880    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.394162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.394140    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.401461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.394429    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.410193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.410144    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv5s7\" (UniqueName: \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.429855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.429772    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.465468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.464085    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\") pod \"auth-test-model-27804d-kserve-565c8dddf7-wzpmn\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") " pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:26.755396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:26.755363    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:27.051931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.051891    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:29:27.054065 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:29:27.054033    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a9c7cab_9831_4633_92b1_9e99aff23b88.slice/crio-aa19b961fe9fedca97dd2f04f3b2d225c2fc0b51867c608b6f63889d45e88f84 WatchSource:0}: Error finding container aa19b961fe9fedca97dd2f04f3b2d225c2fc0b51867c608b6f63889d45e88f84: Status 404 returned error can't find the container with id aa19b961fe9fedca97dd2f04f3b2d225c2fc0b51867c608b6f63889d45e88f84
Oct 30 21:29:27.220500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.219944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:27.851045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.851013    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:29:27.945854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.945809    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" event={"ID":"9a9c7cab-9831-4633-92b1-9e99aff23b88","Type":"ContainerStarted","Data":"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"}
Oct 30 21:29:27.945854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.945856    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" event={"ID":"9a9c7cab-9831-4633-92b1-9e99aff23b88","Type":"ContainerStarted","Data":"aa19b961fe9fedca97dd2f04f3b2d225c2fc0b51867c608b6f63889d45e88f84"}
Oct 30 21:29:27.972639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:27.972571    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" podStartSLOduration=1.972557616 podStartE2EDuration="1.972557616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:27.969361088 +0000 UTC m=+1624.925048316" watchObservedRunningTime="2026-10-30 21:29:27.972557616 +0000 UTC m=+1624.928244884"
Oct 30 21:29:28.404921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:28.404889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:28.756424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:28.756349    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:29.404905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:29.404874    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:30.548770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:30.548738    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:31.674222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:31.674179    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:32.538717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:32.538665    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:33.300277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.300245    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:29:33.307134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.307113    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.317840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.317812    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-46c98c-serving-cert\""
Oct 30 21:29:33.328882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.328853    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:29:33.440103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.440069    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.440269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.440136    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scczn\" (UniqueName: \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.440320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.440284    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.440364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.440319    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.444089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.444062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:33.543647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.543609    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.543871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.543695    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.543871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.543858    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.544000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.543935    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scczn\" (UniqueName: \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.548094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.545398    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.579511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.575478    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.579511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.576064    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.579511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.576095    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scczn\" (UniqueName: \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\") pod \"maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:33.616173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.616137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:33.627798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.627766    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:34.009512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:33.988345    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:29:34.050472 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:29:34.037224    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cf4c78e_ee4d_45ef_93f1_cc1245f2272b.slice/crio-e78e10af7cbd9cabf10554a0922ca823490c8e1721fb7d0f0539470f3b1d066d WatchSource:0}: Error finding container e78e10af7cbd9cabf10554a0922ca823490c8e1721fb7d0f0539470f3b1d066d: Status 404 returned error can't find the container with id e78e10af7cbd9cabf10554a0922ca823490c8e1721fb7d0f0539470f3b1d066d
Oct 30 21:29:34.597575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:34.597532    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:35.066380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.066337    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" event={"ID":"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b","Type":"ContainerStarted","Data":"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"}
Oct 30 21:29:35.066607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.066408    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" event={"ID":"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b","Type":"ContainerStarted","Data":"e78e10af7cbd9cabf10554a0922ca823490c8e1721fb7d0f0539470f3b1d066d"}
Oct 30 21:29:35.066607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.066498    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:35.096853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.096798    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" podStartSLOduration=1.6047544519999999 podStartE2EDuration="2.096783949s" totalImagesPullingTime="492.029497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:35.09276291 +0000 UTC m=+1632.048450139" watchObservedRunningTime="2026-10-30 21:29:35.096783949 +0000 UTC m=+1632.052471176"
Oct 30 21:29:35.606948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.606912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:35.804538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:35.804502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:36.401021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:36.400990    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:36.756332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:36.756263    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:36.773986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:36.773956    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:37.092152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:37.092075    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:29:37.231117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:37.231083    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:38.191688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:38.191642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:39.158530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:39.158494    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:39.976581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:39.975050    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:40.787267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:40.787205    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:41.081204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:41.081124    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:29:41.487591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:41.487549    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:42.148654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:42.148604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:42.961101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:42.961068    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:43.796459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:43.790320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:44.548325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:44.548291    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:45.263102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:45.263072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:45.946111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:45.946071    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:46.830337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:46.830299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:47.706484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:47.706410    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:48.746324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:48.746289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:49.615952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:49.615907    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:49.806010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:49.805969    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:50.925164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:50.925131    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.746920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.746880    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:29:51.753556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.753528    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.762223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.762191    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-46c98c\"/\"openshift-service-ca.crt\""
Oct 30 21:29:51.762898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.762859    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-46c98c\"/\"default-dockercfg-zrtm5\""
Oct 30 21:29:51.763036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.762968    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-46c98c\"/\"autores-model-46c98c-kserve-self-signed-certs\""
Oct 30 21:29:51.763036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.763027    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-46c98c\"/\"kube-root-ca.crt\""
Oct 30 21:29:51.764392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.764365    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:29:51.822191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.822155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.870819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.870786    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.871009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.870885    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.871009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.870916    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wjzk\" (UniqueName: \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.871009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.870997    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.871176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.871098    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.871176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.871132    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.976393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.976360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:51.978297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.978269    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.978521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.978503    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.978660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.978643    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wjzk\" (UniqueName: \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.978826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.978812    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.979063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.979046    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.982785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.982762    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.983455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.983414    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.985556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.983830    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:51.987738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:51.987717    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:52.008844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.008777    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:52.009673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.009606    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:52.010372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.010348    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wjzk\" (UniqueName: \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\") pod \"autores-model-46c98c-kserve-6b7b7fbc77-6fgmx\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") " pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:52.087599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.087559    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:52.463536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.462584    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:29:52.468139 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:29:52.468098    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode856f7a3_33fc_4447_8155_106ce37a333d.slice/crio-663bdef0e45c6eccdfeb4fa5a23f64afb7af05bd942686ad2a3aaa5cf20c6c73 WatchSource:0}: Error finding container 663bdef0e45c6eccdfeb4fa5a23f64afb7af05bd942686ad2a3aaa5cf20c6c73: Status 404 returned error can't find the container with id 663bdef0e45c6eccdfeb4fa5a23f64afb7af05bd942686ad2a3aaa5cf20c6c73
Oct 30 21:29:52.525332 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:29:52.525256    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:29:52.855507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:52.855362    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:53.235332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:53.235289    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" event={"ID":"e856f7a3-33fc-4447-8155-106ce37a333d","Type":"ContainerStarted","Data":"f8ec23b0779a441422ff71f0c26a0bb13cd0d89d2a0fe58e051dfe91fc035c33"}
Oct 30 21:29:53.235886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:53.235346    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" event={"ID":"e856f7a3-33fc-4447-8155-106ce37a333d","Type":"ContainerStarted","Data":"663bdef0e45c6eccdfeb4fa5a23f64afb7af05bd942686ad2a3aaa5cf20c6c73"}
Oct 30 21:29:53.255941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:53.255877    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" podStartSLOduration=2.25586331 podStartE2EDuration="2.25586331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:29:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:29:53.253930103 +0000 UTC m=+1650.209617332" watchObservedRunningTime="2026-10-30 21:29:53.25586331 +0000 UTC m=+1650.211550537"
Oct 30 21:29:53.655811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:53.655775    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:54.088248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:54.088177    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:29:54.437160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:54.437122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:55.242794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:55.242752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:55.968039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:55.968001    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:56.083131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:56.083040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:56.823076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:56.823033    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:57.797277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:57.795978    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:58.608593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:58.603684    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:29:59.508092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:29:59.508043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:00.348572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.348540    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:00.450762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.450671    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:00.450997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.450985    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:00.460001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.459970    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.464776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.464754    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.480411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.479172    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:00.482773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.482747    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lrk2v\""
Oct 30 21:30:00.505195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.505162    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xrjg\" (UniqueName: \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.505529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.505496    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.505713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.505685    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6ckx\" (UniqueName: \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.506134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.506099    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.607615    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xrjg\" (UniqueName: \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.607670    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.607719    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6ckx\" (UniqueName: \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.607948    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.608760    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.608892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.608807    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.654132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.654066    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6ckx\" (UniqueName: \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\") pod \"maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:00.667451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.667402    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xrjg\" (UniqueName: \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.812814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.812787    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:00.831109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:00.830316    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:01.163973 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:30:01.163942    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe0a0478_6fef_4dd1_931c_8d85336f2700.slice/crio-5e2a8d1f587cfec032c3fdb4074a6dc8dcecdea10d5b20bf9cc4a95e5f2937ed WatchSource:0}: Error finding container 5e2a8d1f587cfec032c3fdb4074a6dc8dcecdea10d5b20bf9cc4a95e5f2937ed: Status 404 returned error can't find the container with id 5e2a8d1f587cfec032c3fdb4074a6dc8dcecdea10d5b20bf9cc4a95e5f2937ed
Oct 30 21:30:01.168927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:01.168359    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:01.191068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:01.191045    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:01.191500 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:30:01.191464    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0567f3aa_e0e3_496c_b7fe_7badd8626db8.slice/crio-2e6c13d47950f593ed2c8b56c0d87a98403e4cd249716a46750cecba3d8e50a5 WatchSource:0}: Error finding container 2e6c13d47950f593ed2c8b56c0d87a98403e4cd249716a46750cecba3d8e50a5: Status 404 returned error can't find the container with id 2e6c13d47950f593ed2c8b56c0d87a98403e4cd249716a46750cecba3d8e50a5
Oct 30 21:30:01.292292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:01.292261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:01.305512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:01.305473    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerStarted","Data":"2e6c13d47950f593ed2c8b56c0d87a98403e4cd249716a46750cecba3d8e50a5"}
Oct 30 21:30:01.307336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:01.307297    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerStarted","Data":"5e2a8d1f587cfec032c3fdb4074a6dc8dcecdea10d5b20bf9cc4a95e5f2937ed"}
Oct 30 21:30:02.088457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:02.088039    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:30:02.110234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:02.110204    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:30:02.330754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:02.330727    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:30:02.333257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:02.332872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:03.535511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:03.526540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:04.466579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:04.460928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.282154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:05.282067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:05.359349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:05.359166    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerStarted","Data":"e2f6e806089075b0d096e84fb90ccb17118028af3ddf0b599682e94735189e1d"}
Oct 30 21:30:05.382858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:05.382784    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" podStartSLOduration=1.79436542 podStartE2EDuration="5.382764474s" totalImagesPullingTime="3.588399054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:05.37993762 +0000 UTC m=+1662.335624848" watchObservedRunningTime="2026-10-30 21:30:05.382764474 +0000 UTC m=+1662.338451702"
Oct 30 21:30:06.199582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.198117    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:06.367980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.367945    6738 generic.go:356] "Generic (PLEG): container finished" podID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerID="fe51233c0917d9d8ccd2e104349e9e38c9713d1edb5bc75461b93bd21e4d0bb9" exitCode=60
Oct 30 21:30:06.368466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.368028    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerDied","Data":"fe51233c0917d9d8ccd2e104349e9e38c9713d1edb5bc75461b93bd21e4d0bb9"}
Oct 30 21:30:06.368466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.368423    6738 scope.go:122] "RemoveContainer" containerID="fe51233c0917d9d8ccd2e104349e9e38c9713d1edb5bc75461b93bd21e4d0bb9"
Oct 30 21:30:06.369815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.369795    6738 generic.go:356] "Generic (PLEG): container finished" podID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerID="e2f6e806089075b0d096e84fb90ccb17118028af3ddf0b599682e94735189e1d" exitCode=60
Oct 30 21:30:06.369933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.369833    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerDied","Data":"e2f6e806089075b0d096e84fb90ccb17118028af3ddf0b599682e94735189e1d"}
Oct 30 21:30:06.370137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:06.370121    6738 scope.go:122] "RemoveContainer" containerID="e2f6e806089075b0d096e84fb90ccb17118028af3ddf0b599682e94735189e1d"
Oct 30 21:30:07.048712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:07.048638    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:07.378936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:07.378904    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerStarted","Data":"78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"}
Oct 30 21:30:07.382149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:07.382118    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerStarted","Data":"d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"}
Oct 30 21:30:07.420086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:07.420038    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" podStartSLOduration=3.572811254 podStartE2EDuration="7.42002464s" totalImagesPullingTime="3.847213386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:07.41684036 +0000 UTC m=+1664.372527589" watchObservedRunningTime="2026-10-30 21:30:07.42002464 +0000 UTC m=+1664.375711904"
Oct 30 21:30:07.927783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:07.927728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:08.391798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.391761    6738 generic.go:356] "Generic (PLEG): container finished" podID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerID="d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8" exitCode=60
Oct 30 21:30:08.392275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.391817    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerDied","Data":"d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"}
Oct 30 21:30:08.392275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.391882    6738 scope.go:122] "RemoveContainer" containerID="fe51233c0917d9d8ccd2e104349e9e38c9713d1edb5bc75461b93bd21e4d0bb9"
Oct 30 21:30:08.392275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.392207    6738 scope.go:122] "RemoveContainer" containerID="d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"
Oct 30 21:30:08.392546 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:08.392521    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8_odh-ai-gateway-infra(fe0a0478-6fef-4dd1-931c-8d85336f2700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700"
Oct 30 21:30:08.395139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.395118    6738 generic.go:356] "Generic (PLEG): container finished" podID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerID="78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075" exitCode=60
Oct 30 21:30:08.395244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.395202    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerDied","Data":"78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"}
Oct 30 21:30:08.395429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.395415    6738 scope.go:122] "RemoveContainer" containerID="78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"
Oct 30 21:30:08.395767 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:08.395742    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp_odh-ai-gateway-infra(0567f3aa-e0e3-496c-b7fe-7badd8626db8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8"
Oct 30 21:30:08.406684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.406664    6738 scope.go:122] "RemoveContainer" containerID="e2f6e806089075b0d096e84fb90ccb17118028af3ddf0b599682e94735189e1d"
Oct 30 21:30:08.761108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:08.761019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:09.405571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:09.405536    6738 scope.go:122] "RemoveContainer" containerID="78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"
Oct 30 21:30:09.406050 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:09.405946    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp_odh-ai-gateway-infra(0567f3aa-e0e3-496c-b7fe-7badd8626db8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8"
Oct 30 21:30:09.407569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:09.407545    6738 scope.go:122] "RemoveContainer" containerID="d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"
Oct 30 21:30:09.407877 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:09.407857    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8_odh-ai-gateway-infra(fe0a0478-6fef-4dd1-931c-8d85336f2700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700"
Oct 30 21:30:09.561563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:09.561533    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:10.231550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:10.231509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:10.942670 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:10.942633    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:11.778514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:11.778483    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:12.363880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.363826    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:30:12.384221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.384194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:12.832370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.832295    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"]
Oct 30 21:30:12.838484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.838460    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.842239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.842213    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Oct 30 21:30:12.843997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.843976    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"]
Oct 30 21:30:12.867654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.867619    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-989zl\" (UniqueName: \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.867877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.867856    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.867986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.867890    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.969593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.969555    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-989zl\" (UniqueName: \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.969894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.969868    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.970099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.970072    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.970947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.970924    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.972492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.972465    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:12.978674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:12.978650    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-989zl\" (UniqueName: \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\") pod \"maas-api-revoke-keys-e2e-autores-46c98c-jk485\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:13.049282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:13.049251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:13.156770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:13.156738    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:13.464095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:13.463719    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"]
Oct 30 21:30:13.464540 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:30:13.464286    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3eaeda1_e7eb_43be_96a1_2993c6af704f.slice/crio-850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8 WatchSource:0}: Error finding container 850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8: Status 404 returned error can't find the container with id 850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8
Oct 30 21:30:13.934923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:13.934888    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:14.469243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:14.469207    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485" event={"ID":"d3eaeda1-e7eb-43be-96a1-2993c6af704f","Type":"ContainerStarted","Data":"160b3bc8644c6c68ca7d81c4f274df3d813def9cf12c5ab2fd141bf37975e8a7"}
Oct 30 21:30:14.469687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:14.469265    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485" event={"ID":"d3eaeda1-e7eb-43be-96a1-2993c6af704f","Type":"ContainerStarted","Data":"850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8"}
Oct 30 21:30:14.491662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:14.491608    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485" podStartSLOduration=2.491590366 podStartE2EDuration="2.491590366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:30:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:30:14.489181121 +0000 UTC m=+1671.444868361" watchObservedRunningTime="2026-10-30 21:30:14.491590366 +0000 UTC m=+1671.447277595"
Oct 30 21:30:14.758835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:14.758729    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:15.478161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:15.478130    6738 generic.go:356] "Generic (PLEG): container finished" podID="d3eaeda1-e7eb-43be-96a1-2993c6af704f" containerID="160b3bc8644c6c68ca7d81c4f274df3d813def9cf12c5ab2fd141bf37975e8a7" exitCode=0
Oct 30 21:30:15.478654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:15.478231    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485" event={"ID":"d3eaeda1-e7eb-43be-96a1-2993c6af704f","Type":"ContainerDied","Data":"160b3bc8644c6c68ca7d81c4f274df3d813def9cf12c5ab2fd141bf37975e8a7"}
Oct 30 21:30:15.541581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:15.541542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.259662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.259623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:16.700014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.699987    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:16.739794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.739763    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\") pod \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") "
Oct 30 21:30:16.739940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.739922    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\" (UniqueName: \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\") pod \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") "
Oct 30 21:30:16.740082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.740062    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca" pod "d3eaeda1-e7eb-43be-96a1-2993c6af704f" (UID: "d3eaeda1-e7eb-43be-96a1-2993c6af704f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:16.740082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.740072    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\") pod \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\" (UID: \"d3eaeda1-e7eb-43be-96a1-2993c6af704f\") "
Oct 30 21:30:16.740792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.740770    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-openshift-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:16.742180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.742154    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca" pod "d3eaeda1-e7eb-43be-96a1-2993c6af704f" (UID: "d3eaeda1-e7eb-43be-96a1-2993c6af704f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:16.742270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.742192    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl" pod "d3eaeda1-e7eb-43be-96a1-2993c6af704f" (UID: "d3eaeda1-e7eb-43be-96a1-2993c6af704f"). InnerVolumeSpecName "kube-api-access-989zl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:16.841848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.841766    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-989zl\" (UniqueName: \"kubernetes.io/projected/d3eaeda1-e7eb-43be-96a1-2993c6af704f-kube-api-access-989zl\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:16.841848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:16.841798    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/d3eaeda1-e7eb-43be-96a1-2993c6af704f-maas-api-xks-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:17.181245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:17.181075    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:17.498167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:17.498091    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"
Oct 30 21:30:17.498167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:17.498134    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485" event={"ID":"d3eaeda1-e7eb-43be-96a1-2993c6af704f","Type":"ContainerDied","Data":"850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8"}
Oct 30 21:30:17.498357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:17.498184    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="850fa166034c7b52e864f09d55efdc0c16e8ae4a5a4b8d9c95f75788480134a8"
Oct 30 21:30:17.934928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:17.934887    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:18.156784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:18.156755    6738 scope.go:122] "RemoveContainer" containerID="d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"
Oct 30 21:30:18.743289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:18.743252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.027996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.027957    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:19.157135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.157110    6738 scope.go:122] "RemoveContainer" containerID="78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"
Oct 30 21:30:19.522417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.521501    6738 generic.go:356] "Generic (PLEG): container finished" podID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerID="858d813bb0e627c59aecbaecc6557319618cee50a4c26670cc8f81e86affa9fa" exitCode=60
Oct 30 21:30:19.522417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.521598    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerDied","Data":"858d813bb0e627c59aecbaecc6557319618cee50a4c26670cc8f81e86affa9fa"}
Oct 30 21:30:19.522417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.521673    6738 scope.go:122] "RemoveContainer" containerID="d82241f097958ddd456f1651c5d0265796f74b3453af187625cb7d128fad74f8"
Oct 30 21:30:19.522417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.522083    6738 scope.go:122] "RemoveContainer" containerID="858d813bb0e627c59aecbaecc6557319618cee50a4c26670cc8f81e86affa9fa"
Oct 30 21:30:19.522744 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:19.522462    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8_odh-ai-gateway-infra(fe0a0478-6fef-4dd1-931c-8d85336f2700)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700"
Oct 30 21:30:19.655085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:19.655004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:20.536925 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.536888    6738 generic.go:356] "Generic (PLEG): container finished" podID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerID="4b1641a0dd6ce1bd6362bb79de0885e905f6a05b2875f900373814ad548c1358" exitCode=60
Oct 30 21:30:20.537378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.536922    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerDied","Data":"4b1641a0dd6ce1bd6362bb79de0885e905f6a05b2875f900373814ad548c1358"}
Oct 30 21:30:20.537378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.536998    6738 scope.go:122] "RemoveContainer" containerID="78a46cf1fe0c6e105c591433527520ddcf030690f59e8e08ad8e506438596075"
Oct 30 21:30:20.537719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.537698    6738 scope.go:122] "RemoveContainer" containerID="4b1641a0dd6ce1bd6362bb79de0885e905f6a05b2875f900373814ad548c1358"
Oct 30 21:30:20.538167 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:20.538100    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp_odh-ai-gateway-infra(0567f3aa-e0e3-496c-b7fe-7badd8626db8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8"
Oct 30 21:30:20.585302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.576098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:20.617558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.617029    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:20.836224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.836200    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:20.948537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.947929    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\" (UniqueName: \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\") pod \"fe0a0478-6fef-4dd1-931c-8d85336f2700\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") "
Oct 30 21:30:20.948537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.947980    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\") pod \"fe0a0478-6fef-4dd1-931c-8d85336f2700\" (UID: \"fe0a0478-6fef-4dd1-931c-8d85336f2700\") "
Oct 30 21:30:20.949010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.948738    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca" pod "fe0a0478-6fef-4dd1-931c-8d85336f2700" (UID: "fe0a0478-6fef-4dd1-931c-8d85336f2700"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:20.949281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.949257    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe0a0478-6fef-4dd1-931c-8d85336f2700-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:20.955478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:20.951116    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg" pod "fe0a0478-6fef-4dd1-931c-8d85336f2700" (UID: "fe0a0478-6fef-4dd1-931c-8d85336f2700"). InnerVolumeSpecName "kube-api-access-7xrjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:21.051090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.051008    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xrjg\" (UniqueName: \"kubernetes.io/projected/fe0a0478-6fef-4dd1-931c-8d85336f2700-kube-api-access-7xrjg\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.431757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.431720    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:21.549953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.549909    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8" event={"ID":"fe0a0478-6fef-4dd1-931c-8d85336f2700","Type":"ContainerDied","Data":"5e2a8d1f587cfec032c3fdb4074a6dc8dcecdea10d5b20bf9cc4a95e5f2937ed"}
Oct 30 21:30:21.550418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.549989    6738 scope.go:122] "RemoveContainer" containerID="858d813bb0e627c59aecbaecc6557319618cee50a4c26670cc8f81e86affa9fa"
Oct 30 21:30:21.550418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.550135    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"
Oct 30 21:30:21.575416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.575386    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:21.580273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.580249    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:21.587398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.587215    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-15e79c-29889930-js4k8"]
Oct 30 21:30:21.770984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.770956    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:21.874988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.874957    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\" (UniqueName: \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\") pod \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") "
Oct 30 21:30:21.875171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.875157    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\") pod \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\" (UID: \"0567f3aa-e0e3-496c-b7fe-7badd8626db8\") "
Oct 30 21:30:21.875631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.875592    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca" pod "0567f3aa-e0e3-496c-b7fe-7badd8626db8" (UID: "0567f3aa-e0e3-496c-b7fe-7badd8626db8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:21.876081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.876057    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0567f3aa-e0e3-496c-b7fe-7badd8626db8-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:21.877984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.877953    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx" pod "0567f3aa-e0e3-496c-b7fe-7badd8626db8" (UID: "0567f3aa-e0e3-496c-b7fe-7badd8626db8"). InnerVolumeSpecName "kube-api-access-f6ckx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:21.977716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:21.977684    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6ckx\" (UniqueName: \"kubernetes.io/projected/0567f3aa-e0e3-496c-b7fe-7badd8626db8-kube-api-access-f6ckx\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:22.371524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.371417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:22.515341 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:22.515004    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:30:22.569692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.569663    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"
Oct 30 21:30:22.569692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.569678    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp" event={"ID":"0567f3aa-e0e3-496c-b7fe-7badd8626db8","Type":"ContainerDied","Data":"2e6c13d47950f593ed2c8b56c0d87a98403e4cd249716a46750cecba3d8e50a5"}
Oct 30 21:30:22.570249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.569752    6738 scope.go:122] "RemoveContainer" containerID="4b1641a0dd6ce1bd6362bb79de0885e905f6a05b2875f900373814ad548c1358"
Oct 30 21:30:22.614342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.614308    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:22.619912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.619863    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-a9230f-29889930-8c4sp"]
Oct 30 21:30:22.952751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:22.952715    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:30:23.033426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:23.033395    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:23.167501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:23.167420    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" path="/var/lib/kubelet/pods/0567f3aa-e0e3-496c-b7fe-7badd8626db8/volumes"
Oct 30 21:30:23.167938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:23.167921    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" path="/var/lib/kubelet/pods/fe0a0478-6fef-4dd1-931c-8d85336f2700/volumes"
Oct 30 21:30:23.185619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:23.185591    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:23.406184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:23.406154    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:24.431897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:24.431867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:25.155708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:25.155061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:25.944085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:25.944040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:26.829727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:26.829692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.397964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.397922    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" podUID="e856f7a3-33fc-4447-8155-106ce37a333d" containerName="main" containerID="cri-o://f8ec23b0779a441422ff71f0c26a0bb13cd0d89d2a0fe58e051dfe91fc035c33" gracePeriod=45
Oct 30 21:30:27.596006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.595874    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:27.688106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.688012    6738 generic.go:356] "Generic (PLEG): container finished" podID="e856f7a3-33fc-4447-8155-106ce37a333d" containerID="f8ec23b0779a441422ff71f0c26a0bb13cd0d89d2a0fe58e051dfe91fc035c33" exitCode=0
Oct 30 21:30:27.689722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.689683    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" event={"ID":"e856f7a3-33fc-4447-8155-106ce37a333d","Type":"ContainerDied","Data":"f8ec23b0779a441422ff71f0c26a0bb13cd0d89d2a0fe58e051dfe91fc035c33"}
Oct 30 21:30:27.750635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.750612    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:30:27.821402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821365    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821491    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821561    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\" (UniqueName: \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821633    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821643    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.821890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821794    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821868    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\" (UniqueName: \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\") pod \"e856f7a3-33fc-4447-8155-106ce37a333d\" (UID: \"e856f7a3-33fc-4447-8155-106ce37a333d\") "
Oct 30 21:30:27.821993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.821910    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.826535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.822057    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.826535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.822587    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.826535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.822608    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.826535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.822621    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.826535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.825462    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:27.830502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.827171    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "kube-api-access-2wjzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:27.830502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.827285    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm" pod "e856f7a3-33fc-4447-8155-106ce37a333d" (UID: "e856f7a3-33fc-4447-8155-106ce37a333d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:30:27.924593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.924552    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wjzk\" (UniqueName: \"kubernetes.io/projected/e856f7a3-33fc-4447-8155-106ce37a333d-kube-api-access-2wjzk\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.924593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.924592    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e856f7a3-33fc-4447-8155-106ce37a333d-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.924849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.924607    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e856f7a3-33fc-4447-8155-106ce37a333d-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:27.953870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:27.953785    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" podUID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" containerName="payload-pre-processing" containerID="cri-o://885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4" gracePeriod=25
Oct 30 21:30:28.231973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.231866    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:30:28.285072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.285041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:28.332067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.332014    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\" (UniqueName: \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\") pod \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") "
Oct 30 21:30:28.332273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.332124    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\") pod \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\" (UID: \"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d\") "
Oct 30 21:30:28.332536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.332509    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume" pod "0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" (UID: "0c4e7257-a7bb-403e-94a3-9fb9585f7a6d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:28.333034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.333014    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:28.334845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.334697    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx" pod "0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" (UID: "0c4e7257-a7bb-403e-94a3-9fb9585f7a6d"). InnerVolumeSpecName "kube-api-access-hlrjx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:28.411463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.404193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:28.435474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.435417    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlrjx\" (UniqueName: \"kubernetes.io/projected/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d-kube-api-access-hlrjx\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:28.699235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.699200    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"
Oct 30 21:30:28.699235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.699225    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx" event={"ID":"e856f7a3-33fc-4447-8155-106ce37a333d","Type":"ContainerDied","Data":"663bdef0e45c6eccdfeb4fa5a23f64afb7af05bd942686ad2a3aaa5cf20c6c73"}
Oct 30 21:30:28.699575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.699284    6738 scope.go:122] "RemoveContainer" containerID="f8ec23b0779a441422ff71f0c26a0bb13cd0d89d2a0fe58e051dfe91fc035c33"
Oct 30 21:30:28.701761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.701303    6738 generic.go:356] "Generic (PLEG): container finished" podID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" containerID="885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4" exitCode=0
Oct 30 21:30:28.701761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.701355    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" event={"ID":"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d","Type":"ContainerDied","Data":"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"}
Oct 30 21:30:28.701761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.701359    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"
Oct 30 21:30:28.701761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.701402    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p" event={"ID":"0c4e7257-a7bb-403e-94a3-9fb9585f7a6d","Type":"ContainerDied","Data":"33cb773f5e807bead00096912b1e8847ff87c0e7ffb510c289d2d34b5fb2dfbd"}
Oct 30 21:30:28.715170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.714961    6738 scope.go:122] "RemoveContainer" containerID="885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"
Oct 30 21:30:28.729714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.729686    6738 scope.go:122] "RemoveContainer" containerID="885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"
Oct 30 21:30:28.730126 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:28.730100    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4\": container with ID starting with 885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4 not found: ID does not exist" containerID="885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"
Oct 30 21:30:28.730882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.730137    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4"} err="failed to get container status \"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4\": rpc error: code = NotFound desc = could not find container \"885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4\": container with ID starting with 885853c11c3240ca7cf6f13e7704c4be82fcabb64840d1d691423a3b31389dc4 not found: ID does not exist"
Oct 30 21:30:28.732259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.732235    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:30:28.748601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.748574    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-46c98c/autores-model-46c98c-kserve-6b7b7fbc77-6fgmx"]
Oct 30 21:30:28.753000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.752974    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:30:28.756131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:28.756093    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-46c98c-78cd9c956c-rfh4p"]
Oct 30 21:30:29.049287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:29.049201    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:29.167257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:29.167226    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" path="/var/lib/kubelet/pods/0c4e7257-a7bb-403e-94a3-9fb9585f7a6d/volumes"
Oct 30 21:30:29.167834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:29.167816    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e856f7a3-33fc-4447-8155-106ce37a333d" path="/var/lib/kubelet/pods/e856f7a3-33fc-4447-8155-106ce37a333d/volumes"
Oct 30 21:30:29.867274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:29.867195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:30.645573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:30.645539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:31.459887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:31.459855    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:32.306125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:32.306081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.051809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.051775    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:30:33.053020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.052981    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" podUID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" containerName="maas-api" containerID="cri-o://aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5" gracePeriod=30
Oct 30 21:30:33.122099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.122062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.579661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.543978    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.642082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.642009    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"]
Oct 30 21:30:33.653515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.647154    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-46c98c-jk485"]
Oct 30 21:30:33.695490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.695454    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.736848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.736826    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:30:33.747720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.747681    6738 generic.go:356] "Generic (PLEG): container finished" podID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" containerID="aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5" exitCode=0
Oct 30 21:30:33.747831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.747721    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" event={"ID":"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b","Type":"ContainerDied","Data":"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"}
Oct 30 21:30:33.747831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.747774    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7" event={"ID":"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b","Type":"ContainerDied","Data":"e78e10af7cbd9cabf10554a0922ca823490c8e1721fb7d0f0539470f3b1d066d"}
Oct 30 21:30:33.747831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.747795    6738 scope.go:122] "RemoveContainer" containerID="aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"
Oct 30 21:30:33.747831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.747728    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"
Oct 30 21:30:33.759266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.759246    6738 scope.go:122] "RemoveContainer" containerID="aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"
Oct 30 21:30:33.759575 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:33.759557    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5\": container with ID starting with aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5 not found: ID does not exist" containerID="aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"
Oct 30 21:30:33.759649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.759589    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5"} err="failed to get container status \"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5\": rpc error: code = NotFound desc = could not find container \"aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5\": container with ID starting with aea45471892036974ec8aef826717b0a53579f806b4e3f92104fcbaa13732ca5 not found: ID does not exist"
Oct 30 21:30:33.834701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.834639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:33.880463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.878919    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\") pod \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") "
Oct 30 21:30:33.880463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.878982    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\") pod \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") "
Oct 30 21:30:33.880463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.879118    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\" (UniqueName: \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\") pod \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") "
Oct 30 21:30:33.880463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.879183    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\") pod \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\" (UID: \"1cf4c78e-ee4d-45ef-93f1-cc1245f2272b\") "
Oct 30 21:30:33.884450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.883002    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca" pod "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" (UID: "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:30:33.906196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.906090    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs" pod "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" (UID: "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:33.913966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.910902    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls" pod "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" (UID: "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:30:33.920695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.920618    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn" pod "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" (UID: "1cf4c78e-ee4d-45ef-93f1-cc1245f2272b"). InnerVolumeSpecName "kube-api-access-scczn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:30:33.984060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.984014    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:33.984060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.984051    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:33.984060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.984066    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-scczn\" (UniqueName: \"kubernetes.io/projected/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-kube-api-access-scczn\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:33.984368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:33.984080    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:30:34.079450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:34.077375    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:30:34.082834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:34.082809    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-46c98c-556dd8c89f-lgrq7"]
Oct 30 21:30:34.090341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:34.090282    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:34.662556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:34.662523    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:35.165951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:35.165915    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" path="/var/lib/kubelet/pods/1cf4c78e-ee4d-45ef-93f1-cc1245f2272b/volumes"
Oct 30 21:30:35.166470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:35.166454    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3eaeda1-e7eb-43be-96a1-2993c6af704f" path="/var/lib/kubelet/pods/d3eaeda1-e7eb-43be-96a1-2993c6af704f/volumes"
Oct 30 21:30:35.537299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:35.537227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:36.269558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:36.269515    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:37.030460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:37.030218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:37.845282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:37.845252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:38.603723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:38.603668    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.275852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:39.275825    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:39.968046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:39.968007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.378529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:40.378496    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.501830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:40.501773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.739926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:40.739843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:40.883633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:40.883597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:41.561494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:41.561461    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:42.225879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:42.225812    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:42.983733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:42.983697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:43.718673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:43.718639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:44.603128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:44.603079    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:45.349280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:45.349236    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:46.040291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:46.040251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:46.762347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:46.762308    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:47.523158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:47.523124    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:48.384722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:48.384429    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:49.467457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:49.467396    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:50.426955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:50.426918    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:51.346774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:51.346737    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:52.044076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:52.044038    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:52.511798 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:30:52.511758    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:30:52.725285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:52.725251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:53.475368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:53.475330    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:54.241639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:54.241608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.028752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:55.028719    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.653416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:55.653387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:55.910484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:55.910369    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:56.235793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:56.235670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:56.931639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:56.931608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:57.049491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:57.049423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:57.725294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:57.725261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:58.533695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:58.533616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.455973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:59.455922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.770531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:59.770423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:30:59.899217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:30:59.899167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:00.340170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:00.340139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:00.971607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:00.971571    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.109557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:01.109405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.266968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:01.266860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:01.935044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:01.935016    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:02.602864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:02.602825    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:03.308802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:03.308767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:04.052398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:04.052367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:04.755262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:04.755189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:05.448686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:05.448655    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.113058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:06.113023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:06.833777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:06.833745    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:07.545589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:07.545552    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:08.326579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:08.326530    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:09.026683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:09.026651    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.019201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.019165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.872260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.872228    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:31:10.873595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.873572    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e856f7a3-33fc-4447-8155-106ce37a333d" containerName="main"
Oct 30 21:31:10.873739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.873726    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e856f7a3-33fc-4447-8155-106ce37a333d" containerName="main"
Oct 30 21:31:10.873870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.873859    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.873960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.873950    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874035    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" containerName="payload-pre-processing"
Oct 30 21:31:10.874122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874108    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" containerName="payload-pre-processing"
Oct 30 21:31:10.874211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874200    6738 cpu_manager.go:414] "containerMap: removing container" podUID="d3eaeda1-e7eb-43be-96a1-2993c6af704f" containerName="revoke-keys"
Oct 30 21:31:10.874279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874263    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3eaeda1-e7eb-43be-96a1-2993c6af704f" containerName="revoke-keys"
Oct 30 21:31:10.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874291    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874299    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874329    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874336    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874352    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874359    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874376    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874383    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874394    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874402    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874428    6738 cpu_manager.go:414] "containerMap: removing container" podUID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" containerName="maas-api"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874451    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" containerName="maas-api"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874636    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874658    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874666    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874676    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe0a0478-6fef-4dd1-931c-8d85336f2700" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874706    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874720    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="1cf4c78e-ee4d-45ef-93f1-cc1245f2272b" containerName="maas-api"
Oct 30 21:31:10.874881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.874734    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e856f7a3-33fc-4447-8155-106ce37a333d" containerName="main"
Oct 30 21:31:10.876192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.875597    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c4e7257-a7bb-403e-94a3-9fb9585f7a6d" containerName="payload-pre-processing"
Oct 30 21:31:10.876192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.875616    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3eaeda1-e7eb-43be-96a1-2993c6af704f" containerName="revoke-keys"
Oct 30 21:31:10.886282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.880275    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:10.889787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.889579    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-e596df-serving-cert\""
Oct 30 21:31:10.901222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.901183    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:31:10.942234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.942181    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:10.959154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.959122    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:10.959312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.959161    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dm74\" (UniqueName: \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:10.959430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.959410    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:10.959570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:10.959555    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.063575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.063535    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.064071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.063595    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.064071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.063634    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dm74\" (UniqueName: \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.064071 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:11.063738    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-serving-cert: secret "maas-api-e2e-extmod-a-e596df-serving-cert" not found
Oct 30 21:31:11.064071 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:11.063823    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls podName:23a8f0b7-2960-4dad-a3f6-dc884a011bbe nodeName:}" failed. No retries permitted until 2026-10-30 21:31:11.56378987 +0000 UTC m=+1728.519477096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls") pod "maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe") : secret "maas-api-e2e-extmod-a-e596df-serving-cert" not found
Oct 30 21:31:11.064071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.063950    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.064368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.064282    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.067634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.067608    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.082655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.078370    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dm74\" (UniqueName: \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.082820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.082733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:11.596461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.595731    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:11.596461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:11.596137    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-serving-cert: secret "maas-api-e2e-extmod-a-e596df-serving-cert" not found
Oct 30 21:31:11.596461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:11.596217    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls podName:23a8f0b7-2960-4dad-a3f6-dc884a011bbe nodeName:}" failed. No retries permitted until 2026-10-30 21:31:12.596195393 +0000 UTC m=+1729.551882637 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls") pod "maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe") : secret "maas-api-e2e-extmod-a-e596df-serving-cert" not found
Oct 30 21:31:11.940941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:11.940908    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:12.057459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.057394    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:12.620622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.620586    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:12.624325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.624293    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:12.707055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.707020    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:12.798450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.798280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:12.997794 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:31:12.997760    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a8f0b7_2960_4dad_a3f6_dc884a011bbe.slice/crio-7cd024e0ad5565749b0c868c246d487f64319fbd52ed9d0b9900b29ac7982b6e WatchSource:0}: Error finding container 7cd024e0ad5565749b0c868c246d487f64319fbd52ed9d0b9900b29ac7982b6e: Status 404 returned error can't find the container with id 7cd024e0ad5565749b0c868c246d487f64319fbd52ed9d0b9900b29ac7982b6e
Oct 30 21:31:12.999758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:12.999727    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:31:13.433502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:13.433466    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" event={"ID":"23a8f0b7-2960-4dad-a3f6-dc884a011bbe","Type":"ContainerStarted","Data":"7cd024e0ad5565749b0c868c246d487f64319fbd52ed9d0b9900b29ac7982b6e"}
Oct 30 21:31:13.467416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:13.467374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.177473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:14.177404    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:14.445185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:14.445094    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" event={"ID":"23a8f0b7-2960-4dad-a3f6-dc884a011bbe","Type":"ContainerStarted","Data":"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"}
Oct 30 21:31:14.445185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:14.445151    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:14.469535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:14.469473    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" podStartSLOduration=4.018271752 podStartE2EDuration="4.469456686s" totalImagesPullingTime="451.184934ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:14.463818616 +0000 UTC m=+1731.419505841" watchObservedRunningTime="2026-10-30 21:31:14.469456686 +0000 UTC m=+1731.425143913"
Oct 30 21:31:14.962121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:14.962089    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:15.801978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:15.801924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:16.518678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:16.518643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:17.331624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:17.331589    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:18.133909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:18.133864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:18.862381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:18.862349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.103135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:19.103092    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.208749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:19.208649    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:19.542590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:19.542496    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:20.327068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:20.327020    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:20.459176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:20.459150    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:31:21.053422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:21.053382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:21.897311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:21.897277    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:22.512199 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:22.512149    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:31:22.650955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:22.650919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:23.434800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:23.434768    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.238961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:24.238930    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:24.554423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:24.554343    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:25.177624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:25.177586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:26.038224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:26.038181    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:26.728690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:26.728649    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:27.377489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:27.377453    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.026869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:28.026838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:28.867595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:28.867562    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:29.553468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:29.553407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:30.056069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:30.056034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:30.338957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:30.338839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.003482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:31.003446    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.441233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:31.441202    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.735970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:31.735882    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:31.922176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:31.922145    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.374232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:32.373968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:32.752976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:32.752896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:33.583094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:33.583064    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.318984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:34.318948    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:34.854765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:34.854730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:35.390172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:35.390137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:36.129128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:36.129090    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:36.363491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:36.363453    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:36.517284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:36.517179    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.043589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:37.043560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.582793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:37.582744    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:37.742484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:37.725199    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:38.205216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:38.205187    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:39.040361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:39.040312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:39.881747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:39.881697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:40.562346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:40.562307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:41.266456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:41.266392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:42.179024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:42.178988    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:42.304928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:42.304893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:42.952257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:42.952212    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:31:43.022062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:43.022025    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.151964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:43.151934    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.746957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:43.746919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:43.958220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:43.958182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:44.349464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:44.338568    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:44.720421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:44.720384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:45.049837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:45.049760    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:45.944953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:45.944909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:46.826899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:46.826863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:47.534623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:47.534584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.114476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:48.114408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.267372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:48.267320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:48.389524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:48.387929    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.152051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.152005    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.570972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.570889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.722051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.722016    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.749697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.749667    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:31:49.750494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.750478    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0567f3aa-e0e3-496c-b7fe-7badd8626db8" containerName="cleanup"
Oct 30 21:31:49.754362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.754342    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:49.760497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.760411    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-729802\""
Oct 30 21:31:49.768558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.768533    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:31:49.823008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.822925    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:31:49.830804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.830772    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:49.843149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.842707    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-729802-dockercfg-tv2m9\""
Oct 30 21:31:49.849351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.849333    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:31:49.855291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.855251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:49.933744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.925795    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:49.933744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.925871    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:49.933744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.925901    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ghzj\" (UniqueName: \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:49.933744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:49.926252    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k45p\" (UniqueName: \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.119109    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k45p\" (UniqueName: \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.119205    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.119266    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.119292    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ghzj\" (UniqueName: \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.121205    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.121851    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.141461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.141117    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k45p\" (UniqueName: \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\") pod \"payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.142049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.142019    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ghzj\" (UniqueName: \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\") pod \"payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.173458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.171753    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.394457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.394347    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.501933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.501633    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:31:50.722649 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:31:50.722604    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d14f0bd_4ac5_4a83_bb88_68d6d5828f2a.slice/crio-830ff7225db7fa18d9689243f23a91c562926f6bab695b9d8eeff84502b8abd9 WatchSource:0}: Error finding container 830ff7225db7fa18d9689243f23a91c562926f6bab695b9d8eeff84502b8abd9: Status 404 returned error can't find the container with id 830ff7225db7fa18d9689243f23a91c562926f6bab695b9d8eeff84502b8abd9
Oct 30 21:31:50.725639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.725607    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:31:50.833498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.833468    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" event={"ID":"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8","Type":"ContainerStarted","Data":"d746c9f2542ca18493f71bce14c6f88da92a543074e1fd27d3e850d4b35e51ef"}
Oct 30 21:31:50.833654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.833525    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" event={"ID":"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8","Type":"ContainerStarted","Data":"e5afe2cfbd63e971d4d7c21cf4fb7ff66e82f85073e4de1ccd4d985054307873"}
Oct 30 21:31:50.833654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.833581    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:31:50.835547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.835359    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" event={"ID":"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a","Type":"ContainerStarted","Data":"cfc3d8882f55ee066809be5d659b65bf8ff96bd64bdd3048068136456785f957"}
Oct 30 21:31:50.835547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.835469    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" event={"ID":"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a","Type":"ContainerStarted","Data":"830ff7225db7fa18d9689243f23a91c562926f6bab695b9d8eeff84502b8abd9"}
Oct 30 21:31:50.835547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.835517    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:31:50.855557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.855505    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" podStartSLOduration=1.855487381 podStartE2EDuration="1.855487381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:50.852587993 +0000 UTC m=+1767.808275216" watchObservedRunningTime="2026-10-30 21:31:50.855487381 +0000 UTC m=+1767.811174611"
Oct 30 21:31:50.985078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:50.984954    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:51.666751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.666690    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" podStartSLOduration=2.666671724 podStartE2EDuration="2.666671724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:50.876622883 +0000 UTC m=+1767.832310112" watchObservedRunningTime="2026-10-30 21:31:51.666671724 +0000 UTC m=+1768.622358953"
Oct 30 21:31:51.667997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.667617    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:31:51.677196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.675574    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.686716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.686666    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-f1ed97-serving-cert\""
Oct 30 21:31:51.691814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.691764    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:31:51.761788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.761688    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:51.765334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.765302    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.765573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.765552    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.765622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.765601    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z5sd\" (UniqueName: \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.765762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.765745    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.867494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.867335    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.867494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.867483    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.867765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.867516    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z5sd\" (UniqueName: \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.867765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.867584    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.868272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.868243    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.871427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.871335    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.873397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.873364    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.882520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.882495    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z5sd\" (UniqueName: \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\") pod \"maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:51.905533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:51.905497    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:52.008885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:52.008791    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:52.306879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:52.306834    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:31:52.307746 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:31:52.307700    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efd6bd5_49f0_44e3_9805_c8e437cc4951.slice/crio-2410eaa0ccac9f5551a5c32044032d176649b13ef307876683c1f17a67f1a2db WatchSource:0}: Error finding container 2410eaa0ccac9f5551a5c32044032d176649b13ef307876683c1f17a67f1a2db: Status 404 returned error can't find the container with id 2410eaa0ccac9f5551a5c32044032d176649b13ef307876683c1f17a67f1a2db
Oct 30 21:31:52.601519 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:52.600973    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:31:52.872213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:52.872180    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" event={"ID":"9efd6bd5-49f0-44e3-9805-c8e437cc4951","Type":"ContainerStarted","Data":"2410eaa0ccac9f5551a5c32044032d176649b13ef307876683c1f17a67f1a2db"}
Oct 30 21:31:53.019074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.019045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.438900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.438868    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.663619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.663587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:53.881518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.881482    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" event={"ID":"9efd6bd5-49f0-44e3-9805-c8e437cc4951","Type":"ContainerStarted","Data":"7e596771d5579006e1f6bd0efe57b54c550eb41a6d5ed03bfbe70a7efbb57b2e"}
Oct 30 21:31:53.881899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.881577    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:31:53.901997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:53.901951    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" podStartSLOduration=2.350565837 podStartE2EDuration="2.90193893s" totalImagesPullingTime="551.373093ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:53.899136513 +0000 UTC m=+1770.854823742" watchObservedRunningTime="2026-10-30 21:31:53.90193893 +0000 UTC m=+1770.857626157"
Oct 30 21:31:54.192057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.191974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:54.540482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.540380    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:54.590831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.590795    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:31:54.596955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.596723    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.607887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.607107    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-fb2b58\""
Oct 30 21:31:54.607887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.607215    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-fb2b58-dockercfg-xh76c\""
Oct 30 21:31:54.608776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.608739    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:31:54.714459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.713563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:54.785460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.784418    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.785460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.784522    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnbg5\" (UniqueName: \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.901613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.901067    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.901613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.901156    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnbg5\" (UniqueName: \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.908459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.905759    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.930641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.930579    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnbg5\" (UniqueName: \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\") pod \"payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:54.957707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:54.957679    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:55.132205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.132161    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:31:55.132506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.132424    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy" containerID="cri-o://79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26" gracePeriod=30
Oct 30 21:31:55.277291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.277263    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:31:55.277966 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:31:55.277941    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d70d105_da83_4c2e_aab7_bca9dc599b5f.slice/crio-31077d0ab30aedbe4b89f6207b45a86c99e9daa77054b41e04f2e00f076c735b WatchSource:0}: Error finding container 31077d0ab30aedbe4b89f6207b45a86c99e9daa77054b41e04f2e00f076c735b: Status 404 returned error can't find the container with id 31077d0ab30aedbe4b89f6207b45a86c99e9daa77054b41e04f2e00f076c735b
Oct 30 21:31:55.336784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.336754    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:55.932792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.932738    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" event={"ID":"7d70d105-da83-4c2e-aab7-bca9dc599b5f","Type":"ContainerStarted","Data":"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"}
Oct 30 21:31:55.933249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.932825    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" event={"ID":"7d70d105-da83-4c2e-aab7-bca9dc599b5f","Type":"ContainerStarted","Data":"31077d0ab30aedbe4b89f6207b45a86c99e9daa77054b41e04f2e00f076c735b"}
Oct 30 21:31:55.933249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.932981    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:31:55.951421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:55.951362    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" podStartSLOduration=1.951319156 podStartE2EDuration="1.951319156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:31:55.948323447 +0000 UTC m=+1772.904010674" watchObservedRunningTime="2026-10-30 21:31:55.951319156 +0000 UTC m=+1772.907006385"
Oct 30 21:31:56.065650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:56.065574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:56.862330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:56.862295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.571108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:57.571072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:57.977616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:57.977573    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" podUID="07591128-d54d-45ff-a62d-befd18719dcc" containerName="main" containerID="cri-o://b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016" gracePeriod=45
Oct 30 21:31:58.283366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.283334    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:31:58.409415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.409329    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.409802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.409781    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\" (UniqueName: \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.409957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.409942    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.410186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.410169    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\" (UniqueName: \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.410498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.410372    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.410654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.410636    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\") pod \"07591128-d54d-45ff-a62d-befd18719dcc\" (UID: \"07591128-d54d-45ff-a62d-befd18719dcc\") "
Oct 30 21:31:58.410890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.410871    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:58.412165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.411239    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:58.412165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.411971    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:58.413202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.413156    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.413376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.413338    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.413573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.413560    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.415021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.414998    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:31:58.415387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.415351    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:31:58.425883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.425856    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw" pod "07591128-d54d-45ff-a62d-befd18719dcc" (UID: "07591128-d54d-45ff-a62d-befd18719dcc"). InnerVolumeSpecName "kube-api-access-nmqcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:31:58.515422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.515349    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07591128-d54d-45ff-a62d-befd18719dcc-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.515422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.515385    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07591128-d54d-45ff-a62d-befd18719dcc-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.515422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.515399    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmqcw\" (UniqueName: \"kubernetes.io/projected/07591128-d54d-45ff-a62d-befd18719dcc-kube-api-access-nmqcw\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:31:58.572192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.572157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:58.961057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.961024    6738 generic.go:356] "Generic (PLEG): container finished" podID="07591128-d54d-45ff-a62d-befd18719dcc" containerID="b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016" exitCode=0
Oct 30 21:31:58.961232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.961067    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"
Oct 30 21:31:58.961232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.961109    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" event={"ID":"07591128-d54d-45ff-a62d-befd18719dcc","Type":"ContainerDied","Data":"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"}
Oct 30 21:31:58.961232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.961143    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9" event={"ID":"07591128-d54d-45ff-a62d-befd18719dcc","Type":"ContainerDied","Data":"f55d5892783aa875b73849577475474154f4868bd033bcbb3cbc5ce36f76fcaf"}
Oct 30 21:31:58.961232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.961159    6738 scope.go:122] "RemoveContainer" containerID="b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"
Oct 30 21:31:58.973598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.973577    6738 scope.go:122] "RemoveContainer" containerID="b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"
Oct 30 21:31:58.973871 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:31:58.973849    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016\": container with ID starting with b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016 not found: ID does not exist" containerID="b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"
Oct 30 21:31:58.973948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.973882    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016"} err="failed to get container status \"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016\": rpc error: code = NotFound desc = could not find container \"b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016\": container with ID starting with b4a233d0c3082677a884380f8109eb2b2549bb99f268b62c82fba01e21ff4016 not found: ID does not exist"
Oct 30 21:31:58.991772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.991748    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:31:58.995476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:58.995449    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-06bb0f/auth-test-model-06bb0f-kserve-55f7488fd9-kv6f9"]
Oct 30 21:31:59.072488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:59.072399    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.173488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:59.171211    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07591128-d54d-45ff-a62d-befd18719dcc" path="/var/lib/kubelet/pods/07591128-d54d-45ff-a62d-befd18719dcc/volumes"
Oct 30 21:31:59.855661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:59.855631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:31:59.928950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:31:59.928922    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:32:00.440645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.440610    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:32:00.541424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.541387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:00.549143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549115    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549153    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549307    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549333    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549366    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549381    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:00.549578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549393    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549468    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549598    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:00.549689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549629    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549721    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\") pod \"9208f5f1-f275-4f16-85da-27adcad05ac4\" (UID: \"9208f5f1-f275-4f16-85da-27adcad05ac4\") "
Oct 30 21:32:00.549788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549736    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:00.549974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.549946    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:00.550112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550091    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:00.550480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550461    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.550581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550484    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9208f5f1-f275-4f16-85da-27adcad05ac4-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.550581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550499    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.550581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550511    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.550581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.550525    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.552040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.552012    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:00.552040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.552023    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "kube-api-access-rgplq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:00.552208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.552152    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:00.552406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.552391    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token" pod "9208f5f1-f275-4f16-85da-27adcad05ac4" (UID: "9208f5f1-f275-4f16-85da-27adcad05ac4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:00.651780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.651745    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.651780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.651778    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.651780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.651787    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.652014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.651797    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgplq\" (UniqueName: \"kubernetes.io/projected/9208f5f1-f275-4f16-85da-27adcad05ac4-kube-api-access-rgplq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:00.980916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.980876    6738 generic.go:356] "Generic (PLEG): container finished" podID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerID="79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26" exitCode=0
Oct 30 21:32:00.981343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.980931    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"
Oct 30 21:32:00.981343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.980926    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" event={"ID":"9208f5f1-f275-4f16-85da-27adcad05ac4","Type":"ContainerDied","Data":"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"}
Oct 30 21:32:00.981343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.980988    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts" event={"ID":"9208f5f1-f275-4f16-85da-27adcad05ac4","Type":"ContainerDied","Data":"4ff5c4b76e882abb52733b62f1696e009d3d3e1f905004b6fe091f6fe86a9df4"}
Oct 30 21:32:00.981343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.981011    6738 scope.go:122] "RemoveContainer" containerID="79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"
Oct 30 21:32:00.995160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.995140    6738 scope.go:122] "RemoveContainer" containerID="79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"
Oct 30 21:32:00.995501 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:00.995480    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26\": container with ID starting with 79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26 not found: ID does not exist" containerID="79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"
Oct 30 21:32:00.995590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:00.995518    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26"} err="failed to get container status \"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26\": rpc error: code = NotFound desc = could not find container \"79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26\": container with ID starting with 79cffa7438d9af8e19c27bef273a095c2a5f411f17b4cdb37009571c30825f26 not found: ID does not exist"
Oct 30 21:32:01.023915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.023879    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:32:01.033622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.033601    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-6f9ts"]
Oct 30 21:32:01.165929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.165899    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" path="/var/lib/kubelet/pods/9208f5f1-f275-4f16-85da-27adcad05ac4/volumes"
Oct 30 21:32:01.294854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.294741    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:01.866303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.866272    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:32:01.866499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:01.866348    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:32:02.038107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:02.038076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:02.836187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:02.836139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:03.832294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:03.832248    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:04.929090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929058    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:32:04.929804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929788    6738 cpu_manager.go:414] "containerMap: removing container" podUID="07591128-d54d-45ff-a62d-befd18719dcc" containerName="main"
Oct 30 21:32:04.929804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929805    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07591128-d54d-45ff-a62d-befd18719dcc" containerName="main"
Oct 30 21:32:04.929939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929846    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy"
Oct 30 21:32:04.929939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929851    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy"
Oct 30 21:32:04.929998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929961    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="07591128-d54d-45ff-a62d-befd18719dcc" containerName="main"
Oct 30 21:32:04.929998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.929976    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9208f5f1-f275-4f16-85da-27adcad05ac4" containerName="istio-proxy"
Oct 30 21:32:04.935714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.935648    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:04.945905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.945845    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:32:04.946594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:04.946564    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-729802-serving-cert\""
Oct 30 21:32:05.032739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.032705    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:05.037173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.037148    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.037368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.037350    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.037539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.037522    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s897s\" (UniqueName: \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.037701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.037683    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.147500    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.147675    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s897s\" (UniqueName: \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.147794    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.147862    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:05.148121    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-serving-cert: secret "maas-api-e2e-extmod-a-729802-serving-cert" not found
Oct 30 21:32:05.148460 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:05.148202    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls podName:c9cd5082-c364-4118-83d0-5484a594bd36 nodeName:}" failed. No retries permitted until 2026-10-30 21:32:05.648179877 +0000 UTC m=+1782.603867103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls") pod "maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36") : secret "maas-api-e2e-extmod-a-729802-serving-cert" not found
Oct 30 21:32:05.161462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.149375    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.162168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.162141    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.173513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.173483    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s897s\" (UniqueName: \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.198124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.198050    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:05.665463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:05.660515    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:05.665463 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:05.660983    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-serving-cert: secret "maas-api-e2e-extmod-a-729802-serving-cert" not found
Oct 30 21:32:05.665463 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:05.661046    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls podName:c9cd5082-c364-4118-83d0-5484a594bd36 nodeName:}" failed. No retries permitted until 2026-10-30 21:32:06.661028071 +0000 UTC m=+1783.616715293 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls") pod "maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36") : secret "maas-api-e2e-extmod-a-729802-serving-cert" not found
Oct 30 21:32:06.217003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.216968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.697271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.697232    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:06.700117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.700089    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:06.753305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.753274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:06.763320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.763292    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:06.952886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:06.952799    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:32:07.071595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:07.071564    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:32:07.165113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:07.165080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:07.844580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:07.844503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:08.067126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:08.067087    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" event={"ID":"c9cd5082-c364-4118-83d0-5484a594bd36","Type":"ContainerStarted","Data":"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"}
Oct 30 21:32:08.067652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:08.067159    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" event={"ID":"c9cd5082-c364-4118-83d0-5484a594bd36","Type":"ContainerStarted","Data":"937500f9292c213c811799b11bbbc19d31a598553fcf0deaca67f91b9cff151f"}
Oct 30 21:32:08.067652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:08.067327    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:08.090682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:08.090631    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" podStartSLOduration=3.7108206089999998 podStartE2EDuration="4.090614252s" totalImagesPullingTime="379.793643ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:08.087076532 +0000 UTC m=+1785.042763760" watchObservedRunningTime="2026-10-30 21:32:08.090614252 +0000 UTC m=+1785.046301481"
Oct 30 21:32:08.655169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:08.655132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.563295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:09.563263    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.691224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:09.691189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:09.931810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:09.931767    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:10.627030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:10.626983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:11.303456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:11.303344    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:32:11.348822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:11.348712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:11.473459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:11.467462    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:11.900903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:11.900855    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:12.336345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:12.336258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.038726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:13.038691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.225914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:13.225886    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.466892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:13.466862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.755972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:13.755896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:13.912099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:13.912064    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.079787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:14.079714    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:32:14.341138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:14.338299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.620582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:14.617715    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:14.756607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:14.756569    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:15.562482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:15.562454    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:15.685202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:15.685154    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:16.380939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:16.380901    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:32:16.381169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:16.381146    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy" containerID="cri-o://e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49" gracePeriod=30
Oct 30 21:32:16.643605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:16.643494    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:17.344814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:17.344780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.157402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:18.157369    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.791152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:18.791067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:18.998222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:18.998182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.370745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:19.370714    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.500987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:19.500945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:19.939614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:19.939579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:20.071271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:20.071238    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:20.430956 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:20.430924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.059183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.059153    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.722558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.722521    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:32:21.848866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.848835    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.848938    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.848987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:21.849173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849062    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849090    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849112    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849147    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849151    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:21.849409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849233    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849269    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849317    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:21.849409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849338    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\") pod \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\" (UID: \"039b6249-f141-4c9f-9e6d-3c1e2f297d76\") "
Oct 30 21:32:21.849646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849411    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:21.849646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849416    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:21.849763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.849708    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:21.850524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.850498    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.850524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.850527    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.850734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.850545    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.850734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.850561    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.850734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.850576    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.851940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.851905    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:21.852052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.852010    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:21.852181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.852158    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "kube-api-access-5b8w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:21.852301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.852217    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token" pod "039b6249-f141-4c9f-9e6d-3c1e2f297d76" (UID: "039b6249-f141-4c9f-9e6d-3c1e2f297d76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:21.956599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.956512    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.956599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.956551    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.956599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.956567    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/039b6249-f141-4c9f-9e6d-3c1e2f297d76-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:21.956599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:21.956583    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5b8w2\" (UniqueName: \"kubernetes.io/projected/039b6249-f141-4c9f-9e6d-3c1e2f297d76-kube-api-access-5b8w2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:22.265757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.265677    6738 generic.go:356] "Generic (PLEG): container finished" podID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerID="e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49" exitCode=0
Oct 30 21:32:22.265757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.265724    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"
Oct 30 21:32:22.265960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.265752    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" event={"ID":"039b6249-f141-4c9f-9e6d-3c1e2f297d76","Type":"ContainerDied","Data":"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"}
Oct 30 21:32:22.265960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.265807    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr" event={"ID":"039b6249-f141-4c9f-9e6d-3c1e2f297d76","Type":"ContainerDied","Data":"2ca00add4c2997c6fe160dac87e5ab5f1ba121cf96a0f163c9fc3d3421e5f269"}
Oct 30 21:32:22.265960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.265828    6738 scope.go:122] "RemoveContainer" containerID="e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"
Oct 30 21:32:22.278041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.278021    6738 scope.go:122] "RemoveContainer" containerID="e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"
Oct 30 21:32:22.278340 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:22.278312    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49\": container with ID starting with e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49 not found: ID does not exist" containerID="e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"
Oct 30 21:32:22.278418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.278340    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49"} err="failed to get container status \"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49\": rpc error: code = NotFound desc = could not find container \"e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49\": container with ID starting with e7c85d99860579f87bb31d76132657831c6e68f90cdecd1b6e2b22315f621e49 not found: ID does not exist"
Oct 30 21:32:22.293762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.293741    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:32:22.301066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.301040    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-zqthr"]
Oct 30 21:32:22.517484 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:22.517364    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:32:22.568413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:22.568374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:23.168000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:23.167966    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" path="/var/lib/kubelet/pods/039b6249-f141-4c9f-9e6d-3c1e2f297d76/volumes"
Oct 30 21:32:23.440893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:23.440800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.092795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:24.092758    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.263206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:24.263168    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:24.665478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:24.665428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.583653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:25.541344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:25.722456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:25.716851    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:26.108043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.107959    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:26.348452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.348399    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" podUID="1ec62f94-4382-4259-ac0f-2d961c8f67af" containerName="main" containerID="cri-o://abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242" gracePeriod=45
Oct 30 21:32:26.661345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.661319    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:32:26.768822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.768790    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.768822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.768833    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.769076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.768965    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.769143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769087    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.769143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769126    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\" (UniqueName: \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.769244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769202    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:26.769296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769274    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\") pod \"1ec62f94-4382-4259-ac0f-2d961c8f67af\" (UID: \"1ec62f94-4382-4259-ac0f-2d961c8f67af\") "
Oct 30 21:32:26.769359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769323    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:26.769978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.769949    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:26.770137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.770102    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.770137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.770125    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.771846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.771820    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:26.772305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.772280    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "kube-api-access-k95wm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:26.772570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.772534    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm" pod "1ec62f94-4382-4259-ac0f-2d961c8f67af" (UID: "1ec62f94-4382-4259-ac0f-2d961c8f67af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:26.871927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.871895    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.871927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.871925    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ec62f94-4382-4259-ac0f-2d961c8f67af-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.872119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.871934    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ec62f94-4382-4259-ac0f-2d961c8f67af-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.872119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.871944    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k95wm\" (UniqueName: \"kubernetes.io/projected/1ec62f94-4382-4259-ac0f-2d961c8f67af-kube-api-access-k95wm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:26.964155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:26.964074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:27.324488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.324394    6738 generic.go:356] "Generic (PLEG): container finished" podID="1ec62f94-4382-4259-ac0f-2d961c8f67af" containerID="abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242" exitCode=0
Oct 30 21:32:27.324648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.324492    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"
Oct 30 21:32:27.324648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.324495    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" event={"ID":"1ec62f94-4382-4259-ac0f-2d961c8f67af","Type":"ContainerDied","Data":"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"}
Oct 30 21:32:27.324648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.324604    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp" event={"ID":"1ec62f94-4382-4259-ac0f-2d961c8f67af","Type":"ContainerDied","Data":"30852467efcaf8485dbaf030856ff71983a852b5980e640c6d68bd8d7edce91c"}
Oct 30 21:32:27.324648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.324620    6738 scope.go:122] "RemoveContainer" containerID="abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"
Oct 30 21:32:27.335927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.335905    6738 scope.go:122] "RemoveContainer" containerID="abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"
Oct 30 21:32:27.336219 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:27.336201    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242\": container with ID starting with abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242 not found: ID does not exist" containerID="abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"
Oct 30 21:32:27.336266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.336226    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242"} err="failed to get container status \"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242\": rpc error: code = NotFound desc = could not find container \"abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242\": container with ID starting with abfca61f31de345f49a17af79ba85e65d18505ef62212c43c4ed93d923b09242 not found: ID does not exist"
Oct 30 21:32:27.346912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.346884    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:32:27.354147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.354118    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w4-726a18/auth-test-model-726a18-kserve-84f789cbc7-bpjsp"]
Oct 30 21:32:27.831256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:27.831226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:28.738099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:28.738061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:28.867411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:28.867373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:29.168029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:29.167996    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ec62f94-4382-4259-ac0f-2d961c8f67af" path="/var/lib/kubelet/pods/1ec62f94-4382-4259-ac0f-2d961c8f67af/volumes"
Oct 30 21:32:29.653103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:29.652798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:29.676587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:29.673753    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:32:29.952184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:29.952097    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.224535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:30.224461    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.653726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:30.653690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:30.825715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:30.825558    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:31.718279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:31.718242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:31.826460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:31.825791    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:32:31.826460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:31.826121    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy" containerID="cri-o://85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1" gracePeriod=30
Oct 30 21:32:32.521280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:32.521245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.404372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:33.404339    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:33.964695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:33.964654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:34.430490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:34.430361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:34.728511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:34.723175    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.416931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:35.416901    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.664169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:35.664129    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:35.858117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:35.858026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.058458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:36.056595    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.609457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:36.609045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:36.796697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:36.796658    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:37.231166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.231139    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:32:37.373643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.372086    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.374047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.374020    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.374350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.374331    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.374922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.373125    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:32:37.375025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.374895    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:37.375131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375113    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:37.375333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375310    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.375511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375490    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.375618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375595    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.375839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375787    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.375922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.375891    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.377726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.377696    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:37.378065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.378044    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\") pod \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\" (UID: \"62db81d7-7d94-4f2b-8755-e76ef9f5541f\") "
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.378703    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379026    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379052    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379070    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379084    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.379128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379099    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.379592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.379567    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:37.386230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.386081    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "kube-api-access-mxqwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:37.400466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.396877    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:37.400466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.399619    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo" pod "62db81d7-7d94-4f2b-8755-e76ef9f5541f" (UID: "62db81d7-7d94-4f2b-8755-e76ef9f5541f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:32:37.448505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.447448    6738 generic.go:356] "Generic (PLEG): container finished" podID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerID="85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1" exitCode=0
Oct 30 21:32:37.448505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.447629    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" event={"ID":"62db81d7-7d94-4f2b-8755-e76ef9f5541f","Type":"ContainerDied","Data":"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"}
Oct 30 21:32:37.448505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.447668    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d" event={"ID":"62db81d7-7d94-4f2b-8755-e76ef9f5541f","Type":"ContainerDied","Data":"967dd27adee76aae7e62470f1f84de0bc6a3d093c0ecb7d30160a5e0a558342b"}
Oct 30 21:32:37.448505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.447698    6738 scope.go:122] "RemoveContainer" containerID="85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"
Oct 30 21:32:37.448505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.447828    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"
Oct 30 21:32:37.465156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.465136    6738 scope.go:122] "RemoveContainer" containerID="85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"
Oct 30 21:32:37.465477 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:37.465450    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1\": container with ID starting with 85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1 not found: ID does not exist" containerID="85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"
Oct 30 21:32:37.465537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.465489    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1"} err="failed to get container status \"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1\": rpc error: code = NotFound desc = could not find container \"85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1\": container with ID starting with 85523986e41df81ee75d27ab4abaaecd3965b629ca5a7b3ed38ec9af21149be1 not found: ID does not exist"
Oct 30 21:32:37.476920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.476897    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:32:37.480837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.480810    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.480927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.480847    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.480927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.480864    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.480927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.480880    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxqwf\" (UniqueName: \"kubernetes.io/projected/62db81d7-7d94-4f2b-8755-e76ef9f5541f-kube-api-access-mxqwf\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:37.483190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.483170    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-92n5d"]
Oct 30 21:32:37.676235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:37.676191    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.056315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:38.056283    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:32:38.088951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:38.088875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.634086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:38.632301    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:38.771953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:38.771909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:39.171774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:39.171741    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" path="/var/lib/kubelet/pods/62db81d7-7d94-4f2b-8755-e76ef9f5541f/volumes"
Oct 30 21:32:39.270529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:39.270497    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:39.918704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:39.918660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:40.076461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:40.067374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:40.755933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:40.755903    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.074621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.074527    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.220052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.219994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.289302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.289265    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:32:41.290066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290047    6738 cpu_manager.go:414] "containerMap: removing container" podUID="1ec62f94-4382-4259-ac0f-2d961c8f67af" containerName="main"
Oct 30 21:32:41.290187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290069    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ec62f94-4382-4259-ac0f-2d961c8f67af" containerName="main"
Oct 30 21:32:41.290187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290100    6738 cpu_manager.go:414] "containerMap: removing container" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy"
Oct 30 21:32:41.290187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290105    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy"
Oct 30 21:32:41.290187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290133    6738 cpu_manager.go:414] "containerMap: removing container" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy"
Oct 30 21:32:41.290187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290138    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy"
Oct 30 21:32:41.290495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290282    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ec62f94-4382-4259-ac0f-2d961c8f67af" containerName="main"
Oct 30 21:32:41.290495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290294    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="039b6249-f141-4c9f-9e6d-3c1e2f297d76" containerName="istio-proxy"
Oct 30 21:32:41.290495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.290305    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="62db81d7-7d94-4f2b-8755-e76ef9f5541f" containerName="istio-proxy"
Oct 30 21:32:41.294071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.294050    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.299696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.299668    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-0ea1ea-dockercfg-nmlc4\""
Oct 30 21:32:41.299853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.299668    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-0ea1ea\""
Oct 30 21:32:41.306255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.306229    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.306365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.306338    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xkk\" (UniqueName: \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.315794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.315767    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:32:41.373866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.373835    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.464543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.419612    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.464543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.424424    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.464543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.455869    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xkk\" (UniqueName: \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.470273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.470207    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xkk\" (UniqueName: \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\") pod \"payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.534507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.517252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.620688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.620653    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:41.892806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.892698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:41.987416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:41.987390    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:32:41.994627 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:32:41.994586    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7af83fac_4b27_4d9a_89c5_6aa76508833b.slice/crio-a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff WatchSource:0}: Error finding container a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff: Status 404 returned error can't find the container with id a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff
Oct 30 21:32:42.663456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:42.660555    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" event={"ID":"7af83fac-4b27-4d9a-89c5-6aa76508833b","Type":"ContainerStarted","Data":"1342ce0e5bbb2133e250f8ada57cc567d574458da0a18e87f2962a179dcb4b2a"}
Oct 30 21:32:42.663456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:42.660622    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" event={"ID":"7af83fac-4b27-4d9a-89c5-6aa76508833b","Type":"ContainerStarted","Data":"a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff"}
Oct 30 21:32:42.663456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:42.661672    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:43.178115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:43.178079    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.159351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:44.159319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.284856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:44.284822    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.686487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:44.686457    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:44.719733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:44.719684    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" podUID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" containerName="main" containerID="cri-o://fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045" gracePeriod=45
Oct 30 21:32:45.138013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.137967    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:32:45.215630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.215001    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" podStartSLOduration=4.214985804 podStartE2EDuration="4.214985804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:32:42.707586599 +0000 UTC m=+1819.663273872" watchObservedRunningTime="2026-10-30 21:32:45.214985804 +0000 UTC m=+1822.170673033"
Oct 30 21:32:45.258299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258268    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.258500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258382    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.258500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258494    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.258638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258589    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.258638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258629    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.258759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258662    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\" (UniqueName: \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.258759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258670    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.258759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.258741    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\") pod \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\" (UID: \"9f3926a8-795e-4558-b78e-8a27f6d66ec9\") "
Oct 30 21:32:45.259089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.259064    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.259767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.259407    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.259767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.259476    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.259767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.259493    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.261213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.261190    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:45.261773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.261714    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:45.262203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.262108    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh" pod "9f3926a8-795e-4558-b78e-8a27f6d66ec9" (UID: "9f3926a8-795e-4558-b78e-8a27f6d66ec9"). InnerVolumeSpecName "kube-api-access-wszwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:45.362316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.362208    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f3926a8-795e-4558-b78e-8a27f6d66ec9-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.362316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.362251    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wszwh\" (UniqueName: \"kubernetes.io/projected/9f3926a8-795e-4558-b78e-8a27f6d66ec9-kube-api-access-wszwh\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.362316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.362269    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f3926a8-795e-4558-b78e-8a27f6d66ec9-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:45.374666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.374624    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:45.695249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.695216    6738 generic.go:356] "Generic (PLEG): container finished" podID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" containerID="fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045" exitCode=0
Oct 30 21:32:45.695450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.695264    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"
Oct 30 21:32:45.695450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.695304    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" event={"ID":"9f3926a8-795e-4558-b78e-8a27f6d66ec9","Type":"ContainerDied","Data":"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"}
Oct 30 21:32:45.695450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.695343    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4" event={"ID":"9f3926a8-795e-4558-b78e-8a27f6d66ec9","Type":"ContainerDied","Data":"be96fa0bca7b5512f7259b10f4b84648f464b03fad0933e97216da7efdd8e67d"}
Oct 30 21:32:45.695450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.695360    6738 scope.go:122] "RemoveContainer" containerID="fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"
Oct 30 21:32:45.706932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.706917    6738 scope.go:122] "RemoveContainer" containerID="fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"
Oct 30 21:32:45.707244 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:45.707225    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045\": container with ID starting with fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045 not found: ID does not exist" containerID="fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"
Oct 30 21:32:45.707293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.707254    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045"} err="failed to get container status \"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045\": rpc error: code = NotFound desc = could not find container \"fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045\": container with ID starting with fdf726a0c90176272c2f4a3a2a30526319d6dcd581da430f6c6ca47a26f1b045 not found: ID does not exist"
Oct 30 21:32:45.720852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.720781    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:32:45.724112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:45.724092    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-d4a0a5/auth-test-model-d4a0a5-kserve-68f45b6df-nvxd4"]
Oct 30 21:32:46.366552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:46.366516    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:47.166700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:47.166662    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" path="/var/lib/kubelet/pods/9f3926a8-795e-4558-b78e-8a27f6d66ec9/volumes"
Oct 30 21:32:47.181771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:47.181733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:48.343940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:48.343904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.178186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:49.178145    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:49.373875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:49.373839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:50.048671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:50.048639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:50.814920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:50.814876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:51.583620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:51.583586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:52.517495 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:52.517456    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:32:52.542164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:52.542133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.089375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.088009    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerName="main" containerID="cri-o://e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6" gracePeriod=45
Oct 30 21:32:53.094783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.094748    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerName="main" probeResult="failure" output="Get \"https://10.134.0.102:8000/ready\": EOF"
Oct 30 21:32:53.339748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.339632    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.443531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.443504    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:32:53.459012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.458986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534495    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534698    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534785    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534814    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\" (UniqueName: \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534848    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.545535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.534880    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\") pod \"9a9c7cab-9831-4633-92b1-9e99aff23b88\" (UID: \"9a9c7cab-9831-4633-92b1-9e99aff23b88\") "
Oct 30 21:32:53.609819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.547189    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:53.609819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.548537    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:53.674541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.664397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:53.674541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.673141    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.674541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.673171    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.676627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.676599    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:53.693178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.693119    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:32:53.806463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.796584    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a9c7cab-9831-4633-92b1-9e99aff23b88-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.807265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.807238    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.807455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.801400    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:32:53.919065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.919018    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7" pod "9a9c7cab-9831-4633-92b1-9e99aff23b88" (UID: "9a9c7cab-9831-4633-92b1-9e99aff23b88"). InnerVolumeSpecName "kube-api-access-dv5s7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:32:53.921661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.921620    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a9c7cab-9831-4633-92b1-9e99aff23b88-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.921866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.921849    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dv5s7\" (UniqueName: \"kubernetes.io/projected/9a9c7cab-9831-4633-92b1-9e99aff23b88-kube-api-access-dv5s7\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:32:53.947462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.946393    6738 generic.go:356] "Generic (PLEG): container finished" podID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerID="e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6" exitCode=0
Oct 30 21:32:53.947462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.946576    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" event={"ID":"9a9c7cab-9831-4633-92b1-9e99aff23b88","Type":"ContainerDied","Data":"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"}
Oct 30 21:32:53.947462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.946613    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn" event={"ID":"9a9c7cab-9831-4633-92b1-9e99aff23b88","Type":"ContainerDied","Data":"aa19b961fe9fedca97dd2f04f3b2d225c2fc0b51867c608b6f63889d45e88f84"}
Oct 30 21:32:53.947462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.946637    6738 scope.go:122] "RemoveContainer" containerID="e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"
Oct 30 21:32:53.947462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.946828    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"
Oct 30 21:32:53.985468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:53.983641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.005163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.003532    6738 scope.go:122] "RemoveContainer" containerID="e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"
Oct 30 21:32:54.005163 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:32:54.003884    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6\": container with ID starting with e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6 not found: ID does not exist" containerID="e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"
Oct 30 21:32:54.005163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.003910    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6"} err="failed to get container status \"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6\": rpc error: code = NotFound desc = could not find container \"e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6\": container with ID starting with e9135f5de277da29f42760029737bc3f93aa37050a7b6825b8d3275b54faa9f6 not found: ID does not exist"
Oct 30 21:32:54.008086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.008059    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:32:54.011257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.011228    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w1-27804d/auth-test-model-27804d-kserve-565c8dddf7-wzpmn"]
Oct 30 21:32:54.168408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.168370    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.540128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.540098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.667787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.667650    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:54.687968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:54.687943    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:32:55.166067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:55.166031    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" path="/var/lib/kubelet/pods/9a9c7cab-9831-4633-92b1-9e99aff23b88/volumes"
Oct 30 21:32:55.466983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:55.466900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:56.169519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:56.169490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.028349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:57.028310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:57.750390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:57.750323    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:58.735383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:58.735353    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:59.142497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:59.142427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:32:59.631254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:32:59.631219    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.447350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:00.447314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:00.590464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:00.585998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:01.478265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:01.478229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:02.212377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:02.212344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:02.891277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:02.891238    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:03.744349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:03.739451    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:04.512178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:04.512129    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:05.478627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:05.478585    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:06.156571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:06.156537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:06.864931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:06.864899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:07.594201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:07.594167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:08.358655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:08.358621    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.128295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:09.128265    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:09.954775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:09.954740    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.073120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:10.073053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:10.892213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:10.892127    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:11.528558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:11.528522    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:11.750846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:11.750807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:12.450452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:12.450405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:13.140953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:13.140924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:14.254880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:14.254247    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.056995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:15.056966    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:15.311139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:15.311067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.029026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:16.028997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.234491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:16.234453    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.391958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:16.391912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.529417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:16.529385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:16.843750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:16.843641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:17.021325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:17.021289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:17.181373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:17.181299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:17.625494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:17.625457    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:17.767527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:17.763931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.325861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:18.325829    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:18.542766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:18.542727    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.437604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:19.437561    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:19.553106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:19.553076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:20.311361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:20.311316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:21.226899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:21.226863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.219657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:22.219616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.360361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:22.360314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:22.540310 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:33:22.540222    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:33:23.219061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:23.219024    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:24.050701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:24.050668    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:24.824561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:24.824525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.627461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:25.627406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:25.808009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:25.807947    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:26.342380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:26.342337    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.033271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.033214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.143244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.143211    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.256047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.256019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.399655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.399603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.693134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.693053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.832316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.832261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:27.962094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:27.961989    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.164116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:28.164085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.365098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:28.365054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:28.486003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:28.485895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.287533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.287427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.567767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.567652    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.644999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.644970    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:33:29.645705 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645681    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerName="main"
Oct 30 21:33:29.645705 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645699    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerName="main"
Oct 30 21:33:29.645920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645730    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" containerName="main"
Oct 30 21:33:29.645920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645738    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" containerName="main"
Oct 30 21:33:29.645920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645843    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a9c7cab-9831-4633-92b1-9e99aff23b88" containerName="main"
Oct 30 21:33:29.645920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.645857    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f3926a8-795e-4558-b78e-8a27f6d66ec9" containerName="main"
Oct 30 21:33:29.651301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.651282    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.662583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.662559    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-e2e--8e1d66cd-dockercfg-h75qf\""
Oct 30 21:33:29.662711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.662644    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"openshift-service-ca.crt\""
Oct 30 21:33:29.663163 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.663143    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-42957d623967\""
Oct 30 21:33:29.663272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.663207    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-plugins-e2e-extmod-b-0ea1ea\""
Oct 30 21:33:29.666518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.666497    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"kube-root-ca.crt\""
Oct 30 21:33:29.666912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.666890    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"routing-overlay\""
Oct 30 21:33:29.667962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.667933    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:33:29.789461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.780051    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.789461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.780104    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh72k\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.789461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.780189    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.789461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.780246    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.893216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.893178    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.895049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.895023    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.895636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.895616    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.895820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.895802    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh72k\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.896951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.894931    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.899258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.899223    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.903183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.902564    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.915887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.915833    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh72k\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:29.969874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.969839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:29.971808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:29.971779    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:30.266099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:30.266069    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:33:30.267166 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:33:30.267143    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbdd5db7_fe8b_4adf_8583_7d88ecad8fcb.slice/crio-6c61ab91979176627d70503d6b863e3d704c3f16a6bcf3396c98b50853ba2c64 WatchSource:0}: Error finding container 6c61ab91979176627d70503d6b863e3d704c3f16a6bcf3396c98b50853ba2c64: Status 404 returned error can't find the container with id 6c61ab91979176627d70503d6b863e3d704c3f16a6bcf3396c98b50853ba2c64
Oct 30 21:33:30.277314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:30.277290    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" event={"ID":"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb","Type":"ContainerStarted","Data":"6c61ab91979176627d70503d6b863e3d704c3f16a6bcf3396c98b50853ba2c64"}
Oct 30 21:33:30.335290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:30.335266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.141823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:31.141789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:31.285897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:31.285862    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" event={"ID":"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb","Type":"ContainerStarted","Data":"124c979da4f115a6216e43849dc9d0558ab8e023d4029abc481ff9f0d5e54d34"}
Oct 30 21:33:31.286068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:31.286004    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:31.309360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:31.309301    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" podStartSLOduration=2.30928337 podStartE2EDuration="2.30928337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:31.305319933 +0000 UTC m=+1868.261007183" watchObservedRunningTime="2026-10-30 21:33:31.30928337 +0000 UTC m=+1868.264970598"
Oct 30 21:33:32.023946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:32.023896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.700682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:32.700623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.801338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:32.801297    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:32.942683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:32.942647    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:33.555628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:33.555598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.241723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:34.241686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:34.934724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:34.934691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:35.008044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.008008    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:33:35.013646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.013620    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.024527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.023113    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-02ecd7-openshift-default-dockercfg-drrk7\""
Oct 30 21:33:35.032861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.032815    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:33:35.095075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095028    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095172    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095207    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095336    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095408    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095479    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095556    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095601    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvtnn\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.095672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.095661    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.196853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.196765    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.196853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.196807    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.196898    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.196943    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.196972    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197121    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197213    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197227    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvtnn\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197293    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197367    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197659    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.197942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197921    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.198109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.197998    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.198109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.198090    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.199989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.199961    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.200212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.200194    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.205811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.205785    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvtnn\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.205890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.205872    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\") pod \"e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") " pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.358111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.358083    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:35.651323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.651294    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:33:35.651692 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:33:35.651662    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb223541_5081_46e2_9556_ad61147aa6a4.slice/crio-ecb6c9424d2082828d29de951e6e10f821fef493e72770096b4d115ef9faecac WatchSource:0}: Error finding container ecb6c9424d2082828d29de951e6e10f821fef493e72770096b4d115ef9faecac: Status 404 returned error can't find the container with id ecb6c9424d2082828d29de951e6e10f821fef493e72770096b4d115ef9faecac
Oct 30 21:33:35.654613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.654576    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:35.654721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.654645    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:35.654721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.654687    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:35.771708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:35.771613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:36.067574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.067493    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:33:36.072067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.072037    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.106520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.104723    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.111807    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.111874    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.111948    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzn2\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.111998    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.112069    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.112148    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.112230    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.112278    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.113487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.112325    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.159352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.159324    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-0ea1ea-openshift-default-dockercfg-9tksn\""
Oct 30 21:33:36.213315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213278    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213337    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213387    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213477    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213563    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213618    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzn2\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213677    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213751    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213770    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.213914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213891    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.215138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.213762    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.215138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.214501    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.215138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.214913    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.215941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.215894    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.217551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.217531    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.217812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.217796    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.228225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.228198    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.228419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.228400    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzn2\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.341244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.341166    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" event={"ID":"fb223541-5081-46e2-9556-ad61147aa6a4","Type":"ContainerStarted","Data":"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"}
Oct 30 21:33:36.341413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.341254    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" event={"ID":"fb223541-5081-46e2-9556-ad61147aa6a4","Type":"ContainerStarted","Data":"ecb6c9424d2082828d29de951e6e10f821fef493e72770096b4d115ef9faecac"}
Oct 30 21:33:36.358751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.358727    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:36.360488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.360466    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:36.360583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.360511    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:36.366128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.365674    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podStartSLOduration=2.365657984 podStartE2EDuration="2.365657984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:36.365196447 +0000 UTC m=+1873.320883674" watchObservedRunningTime="2026-10-30 21:33:36.365657984 +0000 UTC m=+1873.321345213"
Oct 30 21:33:36.394419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.394397    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:36.722823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.722774    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:33:36.726972 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:33:36.726946    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdc54a47_5b98_4b0d_b0f5_9628aee9ec50.slice/crio-331c945f175fcd796eabc9e9230adebca855fc0207334337f923fc0c84454336 WatchSource:0}: Error finding container 331c945f175fcd796eabc9e9230adebca855fc0207334337f923fc0c84454336: Status 404 returned error can't find the container with id 331c945f175fcd796eabc9e9230adebca855fc0207334337f923fc0c84454336
Oct 30 21:33:36.730351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.730311    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:36.730477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.730402    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:36.730477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.730469    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:33:36.857409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:36.857316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.131487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.129909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:37.350899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.350858    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" event={"ID":"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50","Type":"ContainerStarted","Data":"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"}
Oct 30 21:33:37.351083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.350926    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" event={"ID":"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50","Type":"ContainerStarted","Data":"331c945f175fcd796eabc9e9230adebca855fc0207334337f923fc0c84454336"}
Oct 30 21:33:37.358926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.358897    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:37.359052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.358947    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:37.385853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.385707    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podStartSLOduration=1.385691092 podStartE2EDuration="1.385691092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:37.380416337 +0000 UTC m=+1874.336103576" watchObservedRunningTime="2026-10-30 21:33:37.385691092 +0000 UTC m=+1874.341378319"
Oct 30 21:33:37.408865 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.408779    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:37.422884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.422566    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:37.422884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.422624    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:37.702535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:37.688167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.359310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.359258    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:38.359762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.359329    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:38.394784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.394751    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:38.395042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.395011    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:38.556073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.556037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:38.748144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:38.748112    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.173464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.168649    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:39.395459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.392098    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:39.395459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.392170    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:39.410961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.410924    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:39.417248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.417214    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:39.736948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:39.736913    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:40.243049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:40.243016    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:40.460708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:40.460672    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:40.461276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:40.461243    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:40.465024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:40.464998    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:40.465254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:40.465212    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:41.365504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:41.363882    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:41.365504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:41.363945    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:41.401553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:41.398875    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:41.401553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:41.398948    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:41.467508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:41.466712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:42.324460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.323620    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:33:42.359582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.359207    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused" start-of-body=
Oct 30 21:33:42.359582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.359252    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.116:15021/healthz/ready\": dial tcp 10.134.0.116:15021: connect: connection refused"
Oct 30 21:33:42.395860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.395827    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:42.396122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.396092    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:42.563509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:42.563471    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:43.363371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.363343    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:43.363947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.363925    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:43.367884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.367851    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:33:43.409459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.406041    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused" start-of-body=
Oct 30 21:33:43.409459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.406091    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.117:15021/healthz/ready\": dial tcp 10.134.0.117:15021: connect: connection refused"
Oct 30 21:33:43.549329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.549294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:43.718518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:43.718121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.402332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.402298    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:44.402654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.402632    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:44.403635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.403613    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:33:44.658969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.658883    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.966831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.966754    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:44.998836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:44.998797    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:33:45.004300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.004242    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.011653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.011590    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:33:45.014002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.013808    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-a378a2\""
Oct 30 21:33:45.033972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.033940    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:33:45.038499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.038479    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.040607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.040587    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:33:45.050510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.045511    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-a378a2-dockercfg-ht7qs\""
Oct 30 21:33:45.090965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.090924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.097545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.097517    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxdr2\" (UniqueName: \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.097963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.097942    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.098167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.098151    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.098316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.098301    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7fq7\" (UniqueName: \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.210707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.200809    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxdr2\" (UniqueName: \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.210707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.201056    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.210707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.201129    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.210707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.201155    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7fq7\" (UniqueName: \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.210707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.202663    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.291514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.203427    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.300941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.300194    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7fq7\" (UniqueName: \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\") pod \"payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.301844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.301815    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxdr2\" (UniqueName: \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\") pod \"payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.368465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.368416    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:45.373721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.373691    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:45.809694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.809599    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:33:45.820700 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:33:45.820662    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod428bc8c4_362d_43af_91ed_69c6e01ee773.slice/crio-5761fcd4c186f5e5bc2860df9f1cbf1dfd2bc46538b832fd8a96e9db1a2ab1c2 WatchSource:0}: Error finding container 5761fcd4c186f5e5bc2860df9f1cbf1dfd2bc46538b832fd8a96e9db1a2ab1c2: Status 404 returned error can't find the container with id 5761fcd4c186f5e5bc2860df9f1cbf1dfd2bc46538b832fd8a96e9db1a2ab1c2
Oct 30 21:33:45.861474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.840182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:45.891948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:45.875076    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:33:46.607458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.598214    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" event={"ID":"74e13002-b907-4fad-ae17-2ff178c1550c","Type":"ContainerStarted","Data":"7879c0c320bee35614ce816b22c97b6d8f7b88c080bb66fa81f7b44398c22d6e"}
Oct 30 21:33:46.607458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.598280    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" event={"ID":"74e13002-b907-4fad-ae17-2ff178c1550c","Type":"ContainerStarted","Data":"96118e2de92288b6366b83cc667ac94cd6b7f0ed63bb352213c97a67d5e2a47e"}
Oct 30 21:33:46.607458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.600038    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:46.647460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.647238    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" event={"ID":"428bc8c4-362d-43af-91ed-69c6e01ee773","Type":"ContainerStarted","Data":"3d2424709325858da11d145b6034c6398785639a5af4051e5816f63b980e5531"}
Oct 30 21:33:46.647460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.647312    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" event={"ID":"428bc8c4-362d-43af-91ed-69c6e01ee773","Type":"ContainerStarted","Data":"5761fcd4c186f5e5bc2860df9f1cbf1dfd2bc46538b832fd8a96e9db1a2ab1c2"}
Oct 30 21:33:46.648041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.647980    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:46.676218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.676159    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" podStartSLOduration=2.67614137 podStartE2EDuration="2.67614137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:46.647638583 +0000 UTC m=+1883.603325811" watchObservedRunningTime="2026-10-30 21:33:46.67614137 +0000 UTC m=+1883.631828599"
Oct 30 21:33:46.733469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.706499    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" podStartSLOduration=2.706479264 podStartE2EDuration="2.706479264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:33:46.67536413 +0000 UTC m=+1883.631051357" watchObservedRunningTime="2026-10-30 21:33:46.706479264 +0000 UTC m=+1883.662166491"
Oct 30 21:33:46.886707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:46.885235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.102468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:47.092933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.222534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:47.219797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.360412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:47.360229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:47.673973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:47.673933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:48.377993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:48.377961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:48.739942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:48.739854    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:33:48.740404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:48.740127    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy" containerID="cri-o://8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb" gracePeriod=30
Oct 30 21:33:49.262044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:49.262004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.000802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:50.000765    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:50.798504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:50.798463    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.040594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:51.040564    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.146842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:51.146809    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.563013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:51.562929    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.694736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:51.694691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:51.824750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:51.824647    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.011611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:52.011577    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.516573 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:33:52.516533    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:33:52.603349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:52.603315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.751192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:52.751157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:52.995180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:52.995132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:53.297310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:53.297230    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:53.402149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:53.402116    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.095261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.095232    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:33:54.199912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.199878    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.199932    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.199961    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200005    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200033    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200058    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200219    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200257    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:54.200359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200277    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200280    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:54.200678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200552    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:54.200678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200583    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\") pod \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\" (UID: \"1f4d5198-328b-4c04-bce2-7202b2d7aa7a\") "
Oct 30 21:33:54.200776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200672    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:33:54.200854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.200812    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:54.201178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.201133    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.201178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.201155    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.201178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.201164    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.201178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.201174    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.201449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.201188    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.202699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.202672    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:33:54.202860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.202846    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:33:54.203045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.203027    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:54.203178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.203068    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9" pod "1f4d5198-328b-4c04-bce2-7202b2d7aa7a" (UID: "1f4d5198-328b-4c04-bce2-7202b2d7aa7a"). InnerVolumeSpecName "kube-api-access-6shh9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:33:54.303937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.303843    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.303937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.303882    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.303937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.303896    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.303937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.303910    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6shh9\" (UniqueName: \"kubernetes.io/projected/1f4d5198-328b-4c04-bce2-7202b2d7aa7a-kube-api-access-6shh9\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:33:54.357601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.357524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.761540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.761506    6738 generic.go:356] "Generic (PLEG): container finished" podID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerID="8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb" exitCode=0
Oct 30 21:33:54.761712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.761587    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"
Oct 30 21:33:54.761712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.761596    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" event={"ID":"1f4d5198-328b-4c04-bce2-7202b2d7aa7a","Type":"ContainerDied","Data":"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"}
Oct 30 21:33:54.761815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.761714    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v" event={"ID":"1f4d5198-328b-4c04-bce2-7202b2d7aa7a","Type":"ContainerDied","Data":"0c3dfe0b3c88843ad3dc55fc54f5b4b31acc99c236eeebda1d9ed72be688e8c6"}
Oct 30 21:33:54.761815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.761748    6738 scope.go:122] "RemoveContainer" containerID="8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"
Oct 30 21:33:54.775090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.775071    6738 scope.go:122] "RemoveContainer" containerID="8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"
Oct 30 21:33:54.775390 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:33:54.775367    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb\": container with ID starting with 8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb not found: ID does not exist" containerID="8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"
Oct 30 21:33:54.775487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.775397    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb"} err="failed to get container status \"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb\": rpc error: code = NotFound desc = could not find container \"8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb\": container with ID starting with 8c5427e9cb480ceee311ff1138aa52d070a46d363ee104dee0f80de3ba9b04fb not found: ID does not exist"
Oct 30 21:33:54.792275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.792232    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:33:54.797425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.797400    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2mz4v"]
Oct 30 21:33:54.859914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.859873    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:54.979573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:54.979543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:55.122097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:55.122062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:55.194459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:55.194201    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" path="/var/lib/kubelet/pods/1f4d5198-328b-4c04-bce2-7202b2d7aa7a/volumes"
Oct 30 21:33:55.198726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:55.197455    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:33:55.198726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:55.197654    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" containerID="cri-o://b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2" gracePeriod=30
Oct 30 21:33:55.243097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:55.243065    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.037869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:56.037795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:56.906460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:56.906371    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.185850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:57.185775    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:57.672360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:57.672327    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:33:57.761476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:57.758330    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.093157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.093070    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.313026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.312992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:58.366911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.366883    6738 patch_prober.go:29] interesting pod/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:33:58.367079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.366953    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:33:58.679577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.679503    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:33:58.838836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:58.838803    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:33:59.709915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:33:59.709872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.502842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.502813    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:34:00.651378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.651349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:00.653608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653587    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653678    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653756    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653779    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653838    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653865    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.653929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653898    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:00.654099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653934    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.654099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653940    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:00.654099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.653997    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.654099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.654070    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:00.654294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.654157    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\") pod \"fb223541-5081-46e2-9556-ad61147aa6a4\" (UID: \"fb223541-5081-46e2-9556-ad61147aa6a4\") "
Oct 30 21:34:00.654294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.654175    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.654568    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.654984    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.655007    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.655022    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fb223541-5081-46e2-9556-ad61147aa6a4-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.655036    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.655301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.655050    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.656521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.656492    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:34:00.656757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.656723    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:34:00.657609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.657584    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:00.680193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.680135    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn" pod "fb223541-5081-46e2-9556-ad61147aa6a4" (UID: "fb223541-5081-46e2-9556-ad61147aa6a4"). InnerVolumeSpecName "kube-api-access-xvtnn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:34:00.757010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.756930    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.757010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.756956    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fb223541-5081-46e2-9556-ad61147aa6a4-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.757010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.756967    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fb223541-5081-46e2-9556-ad61147aa6a4-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:00.757010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:00.756979    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvtnn\" (UniqueName: \"kubernetes.io/projected/fb223541-5081-46e2-9556-ad61147aa6a4-kube-api-access-xvtnn\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:34:01.015663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.015578    6738 generic.go:356] "Generic (PLEG): container finished" podID="fb223541-5081-46e2-9556-ad61147aa6a4" containerID="b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2" exitCode=0
Oct 30 21:34:01.015836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.015765    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" event={"ID":"fb223541-5081-46e2-9556-ad61147aa6a4","Type":"ContainerDied","Data":"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"}
Oct 30 21:34:01.015836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.015815    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd" event={"ID":"fb223541-5081-46e2-9556-ad61147aa6a4","Type":"ContainerDied","Data":"ecb6c9424d2082828d29de951e6e10f821fef493e72770096b4d115ef9faecac"}
Oct 30 21:34:01.015918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.015842    6738 scope.go:122] "RemoveContainer" containerID="b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"
Oct 30 21:34:01.016006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.015993    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"
Oct 30 21:34:01.032850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.032816    6738 scope.go:122] "RemoveContainer" containerID="b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"
Oct 30 21:34:01.033148 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:34:01.033127    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2\": container with ID starting with b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2 not found: ID does not exist" containerID="b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"
Oct 30 21:34:01.033205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.033157    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2"} err="failed to get container status \"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2\": rpc error: code = NotFound desc = could not find container \"b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2\": container with ID starting with b3eb1e07dd8c08988d6aaef9abeae51aeea06b21f2e03473b554f9bf616144b2 not found: ID does not exist"
Oct 30 21:34:01.059218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.059182    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:34:01.070245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.070223    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-02ecd7-openshift-default-6644d8b8c7-k9sjd"]
Oct 30 21:34:01.084181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.084089    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.170222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.170187    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" path="/var/lib/kubelet/pods/fb223541-5081-46e2-9556-ad61147aa6a4/volumes"
Oct 30 21:34:01.220273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.220240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:01.618842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:01.618709    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.240188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.240152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.264918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.264887    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:34:02.265658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265642    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy"
Oct 30 21:34:02.265712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265660    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy"
Oct 30 21:34:02.265712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265691    6738 cpu_manager.go:414] "containerMap: removing container" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy"
Oct 30 21:34:02.265712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265696    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy"
Oct 30 21:34:02.265814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265808    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4d5198-328b-4c04-bce2-7202b2d7aa7a" containerName="istio-proxy"
Oct 30 21:34:02.265849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.265823    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fb223541-5081-46e2-9556-ad61147aa6a4" containerName="istio-proxy"
Oct 30 21:34:02.270872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.270852    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.276301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.276277    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-8072a0\""
Oct 30 21:34:02.276661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.276635    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.276798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.276780    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssfpz\" (UniqueName: \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.278588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.278566    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:34:02.325911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.325764    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:34:02.333498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.333348    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:34:02.333640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.333471    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.342221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.342191    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-8072a0-dockercfg-wq82v\""
Oct 30 21:34:02.370389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.370366    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.378085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.378056    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.378252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.378234    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcwg8\" (UniqueName: \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.378324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.378273    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.379093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.379069    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.379794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.379131    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssfpz\" (UniqueName: \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.387018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.386995    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssfpz\" (UniqueName: \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\") pod \"payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.485459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.482554    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.485459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.482756    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcwg8\" (UniqueName: \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.485459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.484065    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.511897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.508934    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcwg8\" (UniqueName: \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\") pod \"payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.534511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.515695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:02.600460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.599703    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:02.657251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.656514    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:02.947991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.947961    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:34:02.952571 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:34:02.952533    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b67cdf4_bfe7_4b2d_8eee_8e6339a6b836.slice/crio-0f46a52e1f1e81947929710e1cfbf2b42f48b9b82d45fc9992d312c9595bbcdc WatchSource:0}: Error finding container 0f46a52e1f1e81947929710e1cfbf2b42f48b9b82d45fc9992d312c9595bbcdc: Status 404 returned error can't find the container with id 0f46a52e1f1e81947929710e1cfbf2b42f48b9b82d45fc9992d312c9595bbcdc
Oct 30 21:34:02.980616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:02.980590    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:34:02.980744 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:34:02.980721    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad50ef83_4035_47e7_a027_b1e6395290a2.slice/crio-38f7a85972fb26425688c67cd2bc6fb0f21c151487ec7aaaa51abbb47cb21882 WatchSource:0}: Error finding container 38f7a85972fb26425688c67cd2bc6fb0f21c151487ec7aaaa51abbb47cb21882: Status 404 returned error can't find the container with id 38f7a85972fb26425688c67cd2bc6fb0f21c151487ec7aaaa51abbb47cb21882
Oct 30 21:34:03.041752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.041352    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" event={"ID":"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836","Type":"ContainerStarted","Data":"0f46a52e1f1e81947929710e1cfbf2b42f48b9b82d45fc9992d312c9595bbcdc"}
Oct 30 21:34:03.043131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.043105    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" event={"ID":"ad50ef83-4035-47e7-a027-b1e6395290a2","Type":"ContainerStarted","Data":"38f7a85972fb26425688c67cd2bc6fb0f21c151487ec7aaaa51abbb47cb21882"}
Oct 30 21:34:03.240630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.240601    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:03.952877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.952845    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:34:03.958488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.958424    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:03.965135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.965091    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-a378a2-serving-cert\""
Oct 30 21:34:03.970688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:03.970491    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:34:04.034818 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.034784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:04.052676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.052642    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" event={"ID":"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836","Type":"ContainerStarted","Data":"6cfe70996e580afa99cc283d8f339587b93e09f87506cf3456c52e7350b514b1"}
Oct 30 21:34:04.052843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.052707    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:04.054397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.054363    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" event={"ID":"ad50ef83-4035-47e7-a027-b1e6395290a2","Type":"ContainerStarted","Data":"47a183e1f6c282233e7f348a7e77a84e0808e9bf65df9671f5fb5731d62d7738"}
Oct 30 21:34:04.054500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.054423    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:04.072793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.072749    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" podStartSLOduration=2.072736819 podStartE2EDuration="2.072736819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:04.068701546 +0000 UTC m=+1901.024388774" watchObservedRunningTime="2026-10-30 21:34:04.072736819 +0000 UTC m=+1901.028424046"
Oct 30 21:34:04.085767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.085724    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" podStartSLOduration=2.085712893 podStartE2EDuration="2.085712893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:04.084656994 +0000 UTC m=+1901.040344222" watchObservedRunningTime="2026-10-30 21:34:04.085712893 +0000 UTC m=+1901.041400141"
Oct 30 21:34:04.112155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.112121    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r6f2\" (UniqueName: \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.112270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.112206    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.112327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.112303    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.112442 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.112415    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.136957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.136930    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:04.231506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.223228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r6f2\" (UniqueName: \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.231506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.223357    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.231506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.223409    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.231506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.223524    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.231506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.225240    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.246586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.246496    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.252760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.252208    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.257055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.254957    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r6f2\" (UniqueName: \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\") pod \"maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.285298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.285272    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:04.684410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.681297    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:34:04.697310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.697288    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:34:04.928876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:04.928838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:05.064494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:05.064384    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" event={"ID":"e4405c11-ab67-41d8-81ee-0d9690c725bd","Type":"ContainerStarted","Data":"e16c004e3c2f49d1320bce4febef83bd44b4fdab92b9b43603d51d83cdd0ffe2"}
Oct 30 21:34:05.729012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:05.728981    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:06.073963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:06.073878    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" event={"ID":"e4405c11-ab67-41d8-81ee-0d9690c725bd","Type":"ContainerStarted","Data":"a194e624421c03d0fd1aa4342f4b7b70388825225bdc174f4cce0168e04426f0"}
Oct 30 21:34:06.074118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:06.073975    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:06.093008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:06.092960    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" podStartSLOduration=2.573609626 podStartE2EDuration="3.092946198s" totalImagesPullingTime="519.336572ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:06.090745647 +0000 UTC m=+1903.046432873" watchObservedRunningTime="2026-10-30 21:34:06.092946198 +0000 UTC m=+1903.048633424"
Oct 30 21:34:06.441951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:06.441919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:07.255712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:07.255593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.025542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:08.025505    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:08.842177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:08.842137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:09.617018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:09.616984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:10.364841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:10.364789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:11.027341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:11.027312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:11.888084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:11.888051    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:12.086790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:12.086764    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:34:12.631214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:12.631177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:13.321067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:13.321033    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:14.061598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:14.061559    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:14.796764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:14.796733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:15.067128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:15.067059    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:34:15.067128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:15.067127    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:34:15.471399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:15.471368    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:16.150790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:16.150747    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:16.890732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:16.890698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:17.721034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:17.720998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:18.443268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:18.443233    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:18.575477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:18.574877    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:19.442577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:19.442549    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:19.634732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:19.634698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:20.256979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:20.256940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:20.885766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:20.885732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:21.084904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.084866    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:34:21.091124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.091099    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.102287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.102258    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:34:21.234052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.233977    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234026    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234169    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234366    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234401    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234451    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234548    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234670 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234657    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.234728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.234711    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7w74\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.335882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.335855    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.335882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.335886    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.335905    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.335949    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336047    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336098    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7w74\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336138    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336196    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336258    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336271    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336337    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336543    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336646    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.336757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.336749    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.338745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.338719    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.339074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.339051    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.345260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.345224    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.345854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.345833    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7w74\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\") pod \"e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") " pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.427634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.427596    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:21.630829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.630786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:21.722332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.722305    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:34:21.724295 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:34:21.724268    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e02ac66_a892_4357_aba3_c98cdef15dbd.slice/crio-4bed2b504c1cd00ee8372f92c1cdd98f8984988e82d0a7cfa5d11c96734a26f1 WatchSource:0}: Error finding container 4bed2b504c1cd00ee8372f92c1cdd98f8984988e82d0a7cfa5d11c96734a26f1: Status 404 returned error can't find the container with id 4bed2b504c1cd00ee8372f92c1cdd98f8984988e82d0a7cfa5d11c96734a26f1
Oct 30 21:34:21.727596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.727563    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:21.727701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.727626    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:21.727701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:21.727654    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:22.236375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.236323    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" event={"ID":"1e02ac66-a892-4357-aba3-c98cdef15dbd","Type":"ContainerStarted","Data":"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"}
Oct 30 21:34:22.237471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.236393    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" event={"ID":"1e02ac66-a892-4357-aba3-c98cdef15dbd","Type":"ContainerStarted","Data":"4bed2b504c1cd00ee8372f92c1cdd98f8984988e82d0a7cfa5d11c96734a26f1"}
Oct 30 21:34:22.258909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.258858    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podStartSLOduration=1.258840421 podStartE2EDuration="1.258840421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:22.257804184 +0000 UTC m=+1919.213491433" watchObservedRunningTime="2026-10-30 21:34:22.258840421 +0000 UTC m=+1919.214527650"
Oct 30 21:34:22.428776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.428741    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:22.430448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.430413    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:22.430539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.430506    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:22.439759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:22.439711    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:22.573562 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:34:22.573464    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:34:23.428916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:23.428880    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:23.429302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:23.428942    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:23.451691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:23.451622    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:24.331173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:24.319344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:24.428955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:24.428918    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:24.429348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:24.428992    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:24.459796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:24.455600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:25.175033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:25.175003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:25.428174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:25.428091    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:25.428174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:25.428153    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:25.629413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:25.629375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.052778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:26.052059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.220476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:26.213514    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:26.432585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:26.432546    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:26.432901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:26.432869    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:26.785025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:26.784941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:27.052123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:27.052034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:27.429918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:27.429880    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused" start-of-body=
Oct 30 21:34:27.430123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:27.429944    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.123:15021/healthz/ready\": dial tcp 10.134.0.123:15021: connect: connection refused"
Oct 30 21:34:27.942927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:27.942892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:28.434832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:28.434798    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:28.435214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:28.435195    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:28.436058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:28.436031    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:34:28.729320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:28.729232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.565890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:29.565856    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.806318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:29.806276    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:29.922423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:29.922394    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:30.539959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:30.539924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:31.054238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:31.054197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:31.180623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:31.180586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:31.741276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:31.741243    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:32.476585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:32.476543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:33.144311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:33.144274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:33.839587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:33.839554    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:34.656153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:34.656108    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:35.337279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:35.337245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:36.160330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:36.160297    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:36.832613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:36.832570    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:37.522041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:37.522004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:38.235280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:38.235242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:39.097101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:39.097065    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:39.917006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:39.916967    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:40.569391 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:40.569358    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.346449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:41.346397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:41.856575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:41.856535    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:42.346029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:42.345997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:43.141615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:43.141578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:43.282536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:43.282490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:43.694399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:43.694351    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:43.811844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:43.811777    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.081646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:44.081533    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:44.258147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:44.258110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:45.332113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:45.332067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:46.121037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:46.121010    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:46.955182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:46.955149    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:47.818624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:47.818590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:48.391964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.391921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:48.536198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.536140    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:48.751759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.751658    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:34:48.756709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.756680    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:48.770839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.770816    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-f9e5a7-serving-cert\""
Oct 30 21:34:48.773468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.773448    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:34:48.833093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.833064    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:48.915409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.913507    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:48.915860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.915826    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt5sq\" (UniqueName: \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:48.916000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.915900    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:48.916070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.916025    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:48.970228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:48.970199    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:49.018021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.017930    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.018208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.018036    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt5sq\" (UniqueName: \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.018208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.018076    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.018208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.018104    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.018372 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:34:49.018345    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-serving-cert: secret "maas-api-e2e-extmod-b-f9e5a7-serving-cert" not found
Oct 30 21:34:49.018430 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:34:49.018395    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls podName:235355db-f889-4889-a406-d3e371902f52 nodeName:}" failed. No retries permitted until 2026-10-30 21:34:49.518380328 +0000 UTC m=+1946.474067533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls") pod "maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" (UID: "235355db-f889-4889-a406-d3e371902f52") : secret "maas-api-e2e-extmod-b-f9e5a7-serving-cert" not found
Oct 30 21:34:49.018847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.018817    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.021277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.021248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.027133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.027113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt5sq\" (UniqueName: \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.222679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.222634    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:49.530878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.530841    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.558638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.558516    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") pod \"maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.675954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.675921    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:49.854688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.854602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:49.925678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.925645    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:34:49.932871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.932834    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:49.941762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.941723    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-36a099-dockercfg-wt6ff\""
Oct 30 21:34:49.945483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.945462    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:34:49.973619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.973591    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:34:49.983363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.983335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:49.996519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:49.995677    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-36a099\""
Oct 30 21:34:50.055458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.052552    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8c7s\" (UniqueName: \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.055458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.052861    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.165458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.162075    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.165458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.162189    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8c7s\" (UniqueName: \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.168892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.168846    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.197596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.196668    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8c7s\" (UniqueName: \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\") pod \"payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.262767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.262493    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.574057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.574028    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:34:50.575711 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:34:50.575660    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280474f1_ae85_4cbb_81e0_955a58c482b6.slice/crio-e9f49b3d6e9980ccd1e9730f84295698e77e43c1baef1de2e94e812635b9b9b1 WatchSource:0}: Error finding container e9f49b3d6e9980ccd1e9730f84295698e77e43c1baef1de2e94e812635b9b9b1: Status 404 returned error can't find the container with id e9f49b3d6e9980ccd1e9730f84295698e77e43c1baef1de2e94e812635b9b9b1
Oct 30 21:34:50.742199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.742129    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:50.777428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.772088    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" event={"ID":"235355db-f889-4889-a406-d3e371902f52","Type":"ContainerStarted","Data":"ff051bf63441af60195c9e813dcd8f71d087f749c681d0a239a97383882a2c97"}
Oct 30 21:34:50.777428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.772161    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" event={"ID":"235355db-f889-4889-a406-d3e371902f52","Type":"ContainerStarted","Data":"04120f5772b2b215aefa5ae8e1eb89e93e6e566d26aa3e2848425c50ffc8745b"}
Oct 30 21:34:50.777428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.775621    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:50.778803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.778764    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" event={"ID":"280474f1-ae85-4cbb-81e0-955a58c482b6","Type":"ContainerStarted","Data":"f2a88b3baca4bafe0116dd074a09507411b24b0f60816f8cd48a90d5acf44e29"}
Oct 30 21:34:50.778937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.778815    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" event={"ID":"280474f1-ae85-4cbb-81e0-955a58c482b6","Type":"ContainerStarted","Data":"e9f49b3d6e9980ccd1e9730f84295698e77e43c1baef1de2e94e812635b9b9b1"}
Oct 30 21:34:50.783116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.781738    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:34:50.801991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.801933    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" podStartSLOduration=2.380433525 podStartE2EDuration="2.801918707s" totalImagesPullingTime="421.485182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:50.801819708 +0000 UTC m=+1947.757506937" watchObservedRunningTime="2026-10-30 21:34:50.801918707 +0000 UTC m=+1947.757605933"
Oct 30 21:34:50.827193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:50.827136    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" podStartSLOduration=1.827121307 podStartE2EDuration="1.827121307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:50.825608718 +0000 UTC m=+1947.781295947" watchObservedRunningTime="2026-10-30 21:34:50.827121307 +0000 UTC m=+1947.782808533"
Oct 30 21:34:51.508736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:51.508699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:52.229691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:52.229642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:52.510334 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:34:52.510237    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:34:52.860050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:52.859975    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:53.652937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:53.652906    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:54.320254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.320220    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:34:54.329215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.329185    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.338520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.338481    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:34:54.385558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.379560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:54.389905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.386515    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws9r4\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.389905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.386615    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.389905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.386694    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.389905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.386729    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.389905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.386811    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.420937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.420893    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.421158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.421027    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.421158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.421131    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.421325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.421259    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.456578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.456548    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-a378a2-openshift-default-dockercfg-4ch9v\""
Oct 30 21:34:54.523414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523380    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523568    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523624    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523689    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523782    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523816    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523918    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws9r4\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.523999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.523977    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.524111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.524033    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.524111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.524058    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.524111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.524062    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.524902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.524317    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.525047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.524994    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.525253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.525232    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.527379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.527354    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.527782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.527759    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.538386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.538365    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.538690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.538666    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws9r4\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\") pod \"e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") " pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:54.772283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:54.772254    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:55.063821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.063798    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:34:55.067910 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:34:55.067880    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b54311_99ea_4e34_bde6_870e8cfe6a2b.slice/crio-24cfeeaa02d0424563728c3c103e8a632717e00747aa313c1a3125ed9dc5d596 WatchSource:0}: Error finding container 24cfeeaa02d0424563728c3c103e8a632717e00747aa313c1a3125ed9dc5d596: Status 404 returned error can't find the container with id 24cfeeaa02d0424563728c3c103e8a632717e00747aa313c1a3125ed9dc5d596
Oct 30 21:34:55.071864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.071825    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:55.071955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.071915    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:55.071995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.071962    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:34:55.263249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.263217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:55.832524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.832480    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" event={"ID":"a5b54311-99ea-4e34-bde6-870e8cfe6a2b","Type":"ContainerStarted","Data":"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"}
Oct 30 21:34:55.832935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.832567    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" event={"ID":"a5b54311-99ea-4e34-bde6-870e8cfe6a2b","Type":"ContainerStarted","Data":"24cfeeaa02d0424563728c3c103e8a632717e00747aa313c1a3125ed9dc5d596"}
Oct 30 21:34:55.854699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:55.854649    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podStartSLOduration=1.854633398 podStartE2EDuration="1.854633398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:34:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:34:55.850915873 +0000 UTC m=+1952.806603100" watchObservedRunningTime="2026-10-30 21:34:55.854633398 +0000 UTC m=+1952.810320624"
Oct 30 21:34:56.141733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:56.141703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:56.783072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:56.783042    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:34:56.784697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:56.784668    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:34:56.784849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:56.784729    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 21:34:56.821461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:56.816217    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:34:57.059625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:57.059546    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:57.773483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:57.773453    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:34:57.773678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:57.773500    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 21:34:58.071859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:58.071778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:58.770601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:58.769316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:58.773354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:58.773327    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:34:58.773618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:58.773587    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 21:34:59.333227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:59.331943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:59.537462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:59.537396    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:34:59.773548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:59.773506    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:34:59.773739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:34:59.773572    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 21:35:00.375392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:00.375361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:00.773127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:00.773043    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Oct 30 21:35:00.773127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:00.773105    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Oct 30 21:35:01.146130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:01.146082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:01.777591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:01.777562    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:35:01.778051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:01.778035    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:35:01.778685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:01.778663    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:35:01.854460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:01.845158    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:02.560690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:02.560655    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:02.797917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:02.797886    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:35:03.317812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:03.317780    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:03.959656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:03.959620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:04.782272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:04.782237    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:05.434280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:05.434245    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:06.132264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:06.132228    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:06.845329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:06.845294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:07.800301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:07.800258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:07.963324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:07.963249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:08.768463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:08.768422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:09.638902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:09.638861    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:10.570260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:10.570215    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:11.317726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:11.317663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:12.148311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:12.148280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:12.854037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:12.854008    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:13.744088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:13.744054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:14.575509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:14.575477    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:15.577573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:15.577521    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:15.773270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:15.773233    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:16.416412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:16.416367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:16.566662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:16.566613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:16.709561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:16.709424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:17.113881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:17.113061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:17.506953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:17.506863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:17.659465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:17.659398    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:17.861073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:17.861039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.008491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:18.008458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.175910 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:18.175830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.358474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:18.358419    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:18.508103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:18.508019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:19.236717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:19.236685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:19.514540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:19.514428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:20.065057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:20.065023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:20.880368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:20.880335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:21.558457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:21.558411    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:22.513945 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:22.513902    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:35:22.729573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:22.729538    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:23.059490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:23.058022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:23.306470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:23.295325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:24.066453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:24.066402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:25.067198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:25.067162    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:25.728960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:25.728896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:25.892225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:25.892182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:26.056784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:26.056680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:26.266597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:26.265459    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:26.580963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:26.580926    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:27.370128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:27.370096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:27.773991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:27.773891    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:27.971516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:27.971473    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:28.526998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:28.526961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:28.972260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:28.972193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:29.795979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:29.795943    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:35:29.889854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:29.889824    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:30.936139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:30.936087    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:31.671758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:31.671695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:32.273674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.273641    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:35:32.280330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.280308    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.289704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.289682    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-f1ed97-openshift-default-dockercfg-h5n84\""
Oct 30 21:35:32.293844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.293818    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:35:32.377545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377512    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.377700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377618    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.377700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377677    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.377771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377699    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.377771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377737    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.377928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.377909    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.378041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.378027    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.378077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.378054    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqhsm\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.378181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.378168    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479180    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479275    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479297    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqhsm\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479358    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479409    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479525    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479585    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479617    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479636    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479675    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479859    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.479958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.479938    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.480118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.480044    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.480585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.480563    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.482182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.482162    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.482494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.482474    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.494465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.494405    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.494565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.494541    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqhsm\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\") pod \"e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") " pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.598503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.598416    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:32.878475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.878357    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:32.927955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.927925    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:35:32.936311 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:35:32.936268    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod321c2bf4_2dc7_4980_adc5_6d3abc6a8588.slice/crio-d1779c14915433b671b220b31dbe3525c5fcf6b8714e1f7c91cf3fb602580275 WatchSource:0}: Error finding container d1779c14915433b671b220b31dbe3525c5fcf6b8714e1f7c91cf3fb602580275: Status 404 returned error can't find the container with id d1779c14915433b671b220b31dbe3525c5fcf6b8714e1f7c91cf3fb602580275
Oct 30 21:35:32.941098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.941060    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:35:32.941215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.941141    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:35:32.941215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:32.941169    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:35:33.378669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:33.378626    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" event={"ID":"321c2bf4-2dc7-4980-adc5-6d3abc6a8588","Type":"ContainerStarted","Data":"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"}
Oct 30 21:35:33.379124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:33.378727    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" event={"ID":"321c2bf4-2dc7-4980-adc5-6d3abc6a8588","Type":"ContainerStarted","Data":"d1779c14915433b671b220b31dbe3525c5fcf6b8714e1f7c91cf3fb602580275"}
Oct 30 21:35:33.465081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:33.465026    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podStartSLOduration=1.465012089 podStartE2EDuration="1.465012089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:33.463288466 +0000 UTC m=+1990.418975692" watchObservedRunningTime="2026-10-30 21:35:33.465012089 +0000 UTC m=+1990.420699315"
Oct 30 21:35:33.599530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:33.599500    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:33.989576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:33.987790    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:34.622570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:34.618572    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:34.622570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:34.618648    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:34.806459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:34.804890    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" podUID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" containerName="payload-processing" containerID="cri-o://124c979da4f115a6216e43849dc9d0558ab8e023d4029abc481ff9f0d5e54d34" gracePeriod=25
Oct 30 21:35:35.237028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.232022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:35.758654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.758416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:35.781606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.780753    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:35.781606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.780807    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:35.885726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.885688    6738 generic.go:356] "Generic (PLEG): container finished" podID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" containerID="124c979da4f115a6216e43849dc9d0558ab8e023d4029abc481ff9f0d5e54d34" exitCode=0
Oct 30 21:35:35.889976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.886720    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" event={"ID":"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb","Type":"ContainerDied","Data":"124c979da4f115a6216e43849dc9d0558ab8e023d4029abc481ff9f0d5e54d34"}
Oct 30 21:35:35.950161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.949956    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:35:35.973388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.973355    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\") pod \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") "
Oct 30 21:35:35.973550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.973527    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\") pod \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") "
Oct 30 21:35:35.973611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.973600    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\") pod \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") "
Oct 30 21:35:35.973712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.973698    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\") pod \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\" (UID: \"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb\") "
Oct 30 21:35:35.973797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.973736    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume" pod "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" (UID: "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:35.974154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.974128    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume" pod "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" (UID: "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:35:35.976092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.976054    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials" pod "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" (UID: "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:35.976233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.976152    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k" pod "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" (UID: "fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb"). InnerVolumeSpecName "kube-api-access-dh72k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:35:35.976379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.976360    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:35:35.976503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:35.976387    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:35:36.082559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.081129    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh72k\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-kube-api-access-dh72k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:35:36.082559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.081165    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:35:36.206128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.205902    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:36.601663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.601629    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:36.601949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.601921    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:36.849461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.848428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:36.923463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.922867    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"
Oct 30 21:35:36.923463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.922884    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57" event={"ID":"fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb","Type":"ContainerDied","Data":"6c61ab91979176627d70503d6b863e3d704c3f16a6bcf3396c98b50853ba2c64"}
Oct 30 21:35:36.923463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.922946    6738 scope.go:122] "RemoveContainer" containerID="124c979da4f115a6216e43849dc9d0558ab8e023d4029abc481ff9f0d5e54d34"
Oct 30 21:35:36.966103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.966072    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:35:36.975117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:36.975092    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-789796v57"]
Oct 30 21:35:37.044483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:37.044319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:37.165545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:37.165512    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" path="/var/lib/kubelet/pods/fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb/volumes"
Oct 30 21:35:37.513471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:37.512274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:37.603468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:37.602712    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:37.603468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:37.602764    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:38.630467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:38.616855    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:38.630467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:38.616934    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:38.630467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:38.625980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:38.752959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:38.752925    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:38.918512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:38.917850    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:39.475982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:39.470320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:39.599177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:39.599145    6738 patch_prober.go:29] interesting pod/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Oct 30 21:35:39.599349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:39.599219    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Oct 30 21:35:39.649410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:39.649372    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:40.502197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.502147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:40.604248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.604222    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:40.604767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.604744    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:40.605581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.605560    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:35:40.744494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.744417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:40.988964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:40.988932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:41.151529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.151469    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:35:41.174791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.170849    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" containerName="payload-processing"
Oct 30 21:35:41.174791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.170876    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" containerName="payload-processing"
Oct 30 21:35:41.174791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.171112    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fbdd5db7-fe8b-4adf-8583-7d88ecad8fcb" containerName="payload-processing"
Oct 30 21:35:41.189519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.187832    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.204046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.202730    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"routing-overlay\""
Oct 30 21:35:41.204046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.202833    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"agc-cred-c07a671227da\""
Oct 30 21:35:41.204046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.202891    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"payload-processing-external-model-plugins-e2e-extmod-b-f1ed97\""
Oct 30 21:35:41.205050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.204899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:41.227419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.222519    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"payload-processing-external-model-e2e--650b641a-dockercfg-s6bl5\""
Oct 30 21:35:41.267571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.259056    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"kube-root-ca.crt\""
Oct 30 21:35:41.267571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.259159    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"openshift-service-ca.crt\""
Oct 30 21:35:41.308702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.308673    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:35:41.313125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.313094    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg7wf\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.313655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.313629    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.314390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.314371    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.314650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.314631    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.417021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.416988    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.417199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.417027    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.417199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.417066    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.417314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.417210    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg7wf\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.422999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.418996    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.422999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.420088    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.427618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.427569    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.450561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.449889    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg7wf\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.458026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.457996    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:41.558967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.558883    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:41.781716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.776984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:41.915476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.915430    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:35:41.917269 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:35:41.917235    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43daabc_ed85_4390_8cb2_33618be5cd3c.slice/crio-bb04ef6b7c6703433d656023815ef850d4b3fa6c79d1c6560ccf596d561a1009 WatchSource:0}: Error finding container bb04ef6b7c6703433d656023815ef850d4b3fa6c79d1c6560ccf596d561a1009: Status 404 returned error can't find the container with id bb04ef6b7c6703433d656023815ef850d4b3fa6c79d1c6560ccf596d561a1009
Oct 30 21:35:41.967791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:41.967765    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" event={"ID":"f43daabc-ed85-4390-8cb2-33618be5cd3c","Type":"ContainerStarted","Data":"bb04ef6b7c6703433d656023815ef850d4b3fa6c79d1c6560ccf596d561a1009"}
Oct 30 21:35:42.559024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:42.558987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:42.786775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:42.785844    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:42.978157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:42.978116    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" event={"ID":"f43daabc-ed85-4390-8cb2-33618be5cd3c","Type":"ContainerStarted","Data":"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"}
Oct 30 21:35:42.978345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:42.978212    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:43.008538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:43.008480    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" podStartSLOduration=2.008460965 podStartE2EDuration="2.008460965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:43.001669622 +0000 UTC m=+1999.957356850" watchObservedRunningTime="2026-10-30 21:35:43.008460965 +0000 UTC m=+1999.964148196"
Oct 30 21:35:43.063800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:43.063764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:43.283676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:43.283593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:43.487072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:43.487039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:44.478387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:44.478357    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:45.392466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:45.391282    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.006551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.006506    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.181381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.181334    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.405596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.405558    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.550861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.550830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.755673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.755598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:46.931890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:46.931857    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:47.386144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:47.386110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:47.796028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:47.795940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:47.962581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:47.962544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:48.519941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.519900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:48.741787 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.741758    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:35:48.747182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.747157    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.754545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.754521    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-a378a2\"/\"agc-cred-3ca8bb326130\""
Oct 30 21:35:48.758119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.758093    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-a378a2\"/\"payload-processing-external-model-plugins-e2e-extmod-a-a378a2\""
Oct 30 21:35:48.760747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.760721    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-a378a2\"/\"kube-root-ca.crt\""
Oct 30 21:35:48.760871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.760772    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-a378a2\"/\"routing-overlay\""
Oct 30 21:35:48.760871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.760811    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-a378a2\"/\"openshift-service-ca.crt\""
Oct 30 21:35:48.766232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.766206    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:35:48.779226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.779165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:48.854825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.854792    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.854959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.854928    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.855044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.855028    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.855124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.855110    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn4kb\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.956861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.956815    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.957018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.956924    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.957018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.956993    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn4kb\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.957259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.957232    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.957694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.957669    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.957694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.957682    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.959828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.959804    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.969636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.969614    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn4kb\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\") pod \"payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") " pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:48.978567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:48.978544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:49.088508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:49.087782    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:49.321623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:49.300449    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:49.504886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:49.504838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:50.012554 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:35:50.012516    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda28a3c0e_be50_4c5b_98a8_ca40868f47ab.slice/crio-fa551715dcbc02e9ec83b751e485a8c9f75b7a79e5f7ec3974c707d3e7e4fdb8 WatchSource:0}: Error finding container fa551715dcbc02e9ec83b751e485a8c9f75b7a79e5f7ec3974c707d3e7e4fdb8: Status 404 returned error can't find the container with id fa551715dcbc02e9ec83b751e485a8c9f75b7a79e5f7ec3974c707d3e7e4fdb8
Oct 30 21:35:50.015622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.015594    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:35:50.291165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.291080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:50.292812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.292783    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" event={"ID":"a28a3c0e-be50-4c5b-98a8-ca40868f47ab","Type":"ContainerStarted","Data":"dcb10413635ce299781be38729db15a4d4502cf9379ab821ebbff57473a7c3c1"}
Oct 30 21:35:50.292953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.292861    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:35:50.292953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.292885    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" event={"ID":"a28a3c0e-be50-4c5b-98a8-ca40868f47ab","Type":"ContainerStarted","Data":"fa551715dcbc02e9ec83b751e485a8c9f75b7a79e5f7ec3974c707d3e7e4fdb8"}
Oct 30 21:35:50.332127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.332070    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" podStartSLOduration=2.332055651 podStartE2EDuration="2.332055651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:35:50.328760405 +0000 UTC m=+2007.284447634" watchObservedRunningTime="2026-10-30 21:35:50.332055651 +0000 UTC m=+2007.287742879"
Oct 30 21:35:50.757464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:50.749413    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:51.028266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:51.028057    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:51.254957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:51.254926    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:51.371540 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.368122    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:51.371540 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.368144    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:51.371540 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.368195    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:35:51.868180566 +0000 UTC m=+2008.823867785 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:51.760700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:51.760616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:51.877235 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.877203    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:51.877235 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.877227    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:51.877505 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:51.877291    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:35:52.877271272 +0000 UTC m=+2009.832958498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:52.509829 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:52.509788    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:35:52.850392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:52.850314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:52.896027 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:52.895996    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:52.896027 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:52.896023    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:52.896236 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:52.896086    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:35:54.896066796 +0000 UTC m=+2011.851754003 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:53.626075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:53.625987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:53.905332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:53.905227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:53.991464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:53.991420    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:35:54.087045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:54.086973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:54.264988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:54.264903    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:54.421804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:54.421773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:54.727114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:54.727033    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:54.941235 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:54.941178    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:54.941554 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:54.941429    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:54.941788 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:54.941769    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:35:58.94175349 +0000 UTC m=+2015.897440709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:55.043133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:55.043041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:55.270069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:55.270033    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:56.319732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:56.319669    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:57.434470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:57.433547    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:58.212919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:58.212881    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:59.008842 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:59.008793    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:59.008842 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:59.008816    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:59.009288 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:35:59.008890    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:36:07.008876845 +0000 UTC m=+2023.964564051 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:35:59.059143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:59.059107    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:35:59.929463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:35:59.924315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:00.639860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:00.639827    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:01.305381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:01.305353    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:36:01.456407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:01.456374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:02.268997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:02.268967    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:03.244273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:03.244229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:03.939610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:03.939581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:04.273964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:04.273887    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:36:04.875940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:04.875895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:05.756686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:05.756642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:06.541019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:06.540973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:07.043519 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:07.043485    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-a378a2/agc-cred-3ca8bb326130: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:36:07.043519 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:07.043513    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr: secret "agc-cred-3ca8bb326130" not found
Oct 30 21:36:07.043952 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:07.043582    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials podName:a28a3c0e-be50-4c5b-98a8-ca40868f47ab nodeName:}" failed. No retries permitted until 2026-10-30 21:36:23.043568305 +0000 UTC m=+2039.999255536 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab") : secret "agc-cred-3ca8bb326130" not found
Oct 30 21:36:07.254127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:07.254096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:07.956575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:07.956534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:08.263513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:08.263403    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:36:08.734696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:08.734663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:09.274225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.274185    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" podUID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" containerName="payload-processing" containerID="cri-o://dcb10413635ce299781be38729db15a4d4502cf9379ab821ebbff57473a7c3c1" gracePeriod=25
Oct 30 21:36:09.445419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.445383    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:09.569800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.569393    6738 generic.go:356] "Generic (PLEG): container finished" podID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" containerID="dcb10413635ce299781be38729db15a4d4502cf9379ab821ebbff57473a7c3c1" exitCode=0
Oct 30 21:36:09.569800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.569627    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" event={"ID":"a28a3c0e-be50-4c5b-98a8-ca40868f47ab","Type":"ContainerDied","Data":"dcb10413635ce299781be38729db15a4d4502cf9379ab821ebbff57473a7c3c1"}
Oct 30 21:36:09.648111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.648085    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:36:09.699589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.699557    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\") pod \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") "
Oct 30 21:36:09.699798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.699642    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\") pod \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") "
Oct 30 21:36:09.699896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.699870    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\") pod \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") "
Oct 30 21:36:09.700151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.700109    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\") pod \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\" (UID: \"a28a3c0e-be50-4c5b-98a8-ca40868f47ab\") "
Oct 30 21:36:09.700569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.700542    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume" pod "a28a3c0e-be50-4c5b-98a8-ca40868f47ab" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:09.700985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.700960    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:09.701106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.701022    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume" pod "a28a3c0e-be50-4c5b-98a8-ca40868f47ab" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:09.703340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.703319    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb" pod "a28a3c0e-be50-4c5b-98a8-ca40868f47ab" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab"). InnerVolumeSpecName "kube-api-access-kn4kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:09.703451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.703350    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials" pod "a28a3c0e-be50-4c5b-98a8-ca40868f47ab" (UID: "a28a3c0e-be50-4c5b-98a8-ca40868f47ab"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:09.802727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.802689    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:09.802727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.802720    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:09.802727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:09.802730    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn4kb\" (UniqueName: \"kubernetes.io/projected/a28a3c0e-be50-4c5b-98a8-ca40868f47ab-kube-api-access-kn4kb\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:10.647634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.643300    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr" event={"ID":"a28a3c0e-be50-4c5b-98a8-ca40868f47ab","Type":"ContainerDied","Data":"fa551715dcbc02e9ec83b751e485a8c9f75b7a79e5f7ec3974c707d3e7e4fdb8"}
Oct 30 21:36:10.647634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.643376    6738 scope.go:122] "RemoveContainer" containerID="dcb10413635ce299781be38729db15a4d4502cf9379ab821ebbff57473a7c3c1"
Oct 30 21:36:10.647634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.643568    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"
Oct 30 21:36:10.817474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.817421    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:36:10.820918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.820840    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:10.839889 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:10.839833    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-a378a2/payload-processing-external-model-e2e-extmod-a-a378a2-7f7b9cdrr"]
Oct 30 21:36:11.268460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:11.267980    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" path="/var/lib/kubelet/pods/a28a3c0e-be50-4c5b-98a8-ca40868f47ab/volumes"
Oct 30 21:36:12.094558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:12.094280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:13.266461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.266401    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" podUID="f43daabc-ed85-4390-8cb2-33618be5cd3c" containerName="payload-processing" containerID="cri-o://d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21" gracePeriod=25
Oct 30 21:36:13.269467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.269424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:13.625494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.625421    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:36:13.681893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.681859    6738 generic.go:356] "Generic (PLEG): container finished" podID="f43daabc-ed85-4390-8cb2-33618be5cd3c" containerID="d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21" exitCode=0
Oct 30 21:36:13.682149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.682123    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"
Oct 30 21:36:13.682149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.682042    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" event={"ID":"f43daabc-ed85-4390-8cb2-33618be5cd3c","Type":"ContainerDied","Data":"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"}
Oct 30 21:36:13.682301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.682210    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r" event={"ID":"f43daabc-ed85-4390-8cb2-33618be5cd3c","Type":"ContainerDied","Data":"bb04ef6b7c6703433d656023815ef850d4b3fa6c79d1c6560ccf596d561a1009"}
Oct 30 21:36:13.682301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.682231    6738 scope.go:122] "RemoveContainer" containerID="d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"
Oct 30 21:36:13.694895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.694873    6738 scope.go:122] "RemoveContainer" containerID="d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"
Oct 30 21:36:13.695229 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:13.695205    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21\": container with ID starting with d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21 not found: ID does not exist" containerID="d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"
Oct 30 21:36:13.695310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.695241    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21"} err="failed to get container status \"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21\": rpc error: code = NotFound desc = could not find container \"d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21\": container with ID starting with d193455698f9b718b29dac62e9b07c5fba8e7a946c30358091ca308b2fbe3a21 not found: ID does not exist"
Oct 30 21:36:13.767089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767057    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\") pod \"f43daabc-ed85-4390-8cb2-33618be5cd3c\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") "
Oct 30 21:36:13.767330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767155    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\") pod \"f43daabc-ed85-4390-8cb2-33618be5cd3c\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") "
Oct 30 21:36:13.767330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767268    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\") pod \"f43daabc-ed85-4390-8cb2-33618be5cd3c\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") "
Oct 30 21:36:13.767484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767462    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\") pod \"f43daabc-ed85-4390-8cb2-33618be5cd3c\" (UID: \"f43daabc-ed85-4390-8cb2-33618be5cd3c\") "
Oct 30 21:36:13.767933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767845    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume" pod "f43daabc-ed85-4390-8cb2-33618be5cd3c" (UID: "f43daabc-ed85-4390-8cb2-33618be5cd3c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:13.767933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.767896    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume" pod "f43daabc-ed85-4390-8cb2-33618be5cd3c" (UID: "f43daabc-ed85-4390-8cb2-33618be5cd3c"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:13.768264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.768242    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:13.768353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.768267    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f43daabc-ed85-4390-8cb2-33618be5cd3c-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:13.772481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.770247    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf" pod "f43daabc-ed85-4390-8cb2-33618be5cd3c" (UID: "f43daabc-ed85-4390-8cb2-33618be5cd3c"). InnerVolumeSpecName "kube-api-access-vg7wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:13.772481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.770289    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials" pod "f43daabc-ed85-4390-8cb2-33618be5cd3c" (UID: "f43daabc-ed85-4390-8cb2-33618be5cd3c"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:13.869871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.869837    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg7wf\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-kube-api-access-vg7wf\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:13.870022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:13.869874    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f43daabc-ed85-4390-8cb2-33618be5cd3c-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:14.027229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.027191    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:36:14.035399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.035374    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-b754j927r"]
Oct 30 21:36:14.122497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.122393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:14.353909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.353872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:14.575124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.575031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:14.823456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:14.823356    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:15.169072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:15.169035    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f43daabc-ed85-4390-8cb2-33618be5cd3c" path="/var/lib/kubelet/pods/f43daabc-ed85-4390-8cb2-33618be5cd3c/volumes"
Oct 30 21:36:15.962092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:15.962056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:16.464383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.464349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:16.545456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.545411    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:16.546290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546269    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" containerName="payload-processing"
Oct 30 21:36:16.546429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546413    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" containerName="payload-processing"
Oct 30 21:36:16.546631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546615    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f43daabc-ed85-4390-8cb2-33618be5cd3c" containerName="payload-processing"
Oct 30 21:36:16.546707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546632    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f43daabc-ed85-4390-8cb2-33618be5cd3c" containerName="payload-processing"
Oct 30 21:36:16.546866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546851    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f43daabc-ed85-4390-8cb2-33618be5cd3c" containerName="payload-processing"
Oct 30 21:36:16.546930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.546880    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a28a3c0e-be50-4c5b-98a8-ca40868f47ab" containerName="payload-processing"
Oct 30 21:36:16.550705 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.550625    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.558110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.558089    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"payload-processing-external-model-plugins-e2e-extmod-a-fb2b58\""
Oct 30 21:36:16.558222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.558164    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"openshift-service-ca.crt\""
Oct 30 21:36:16.561126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.561105    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"agc-cred-edf96af6b691\""
Oct 30 21:36:16.562986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.562961    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.563122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.563103    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.564256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.563348    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.564256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.563421    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s86zt\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.564256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.563702    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"payload-processing-external-model-e2e--b5c8139c-dockercfg-ch2nk\""
Oct 30 21:36:16.569146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.569128    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.674972    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"kube-root-ca.crt\""
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.675313    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"routing-overlay\""
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.680386    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.680479    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s86zt\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.680631    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.680736    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.690614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.689248    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.706281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.698416    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.711307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.709146    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.728526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.723144    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s86zt\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:16.898471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:16.897829    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:17.066336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.066298    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:36:17.075469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.073945    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.085973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.085950    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-42957d623967\""
Oct 30 21:36:17.086801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.086783    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-11235e5f2a92\""
Oct 30 21:36:17.087147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087116    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-d390aee96ac0\""
Oct 30 21:36:17.087147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087118    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"openshift-service-ca.crt\""
Oct 30 21:36:17.087147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087142    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-e2e--8e1d66cd-dockercfg-h75qf\""
Oct 30 21:36:17.087295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087140    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-plugins-e2e-extmod-b-0ea1ea\""
Oct 30 21:36:17.087295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087180    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"routing-overlay\""
Oct 30 21:36:17.087426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.087406    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"kube-root-ca.crt\""
Oct 30 21:36:17.122112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.122072    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.122284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.122227    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.122284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.122254    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.122484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.122303    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjpm2\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.123895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.123868    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:36:17.223668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.223631    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.223900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.223870    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.223974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.223922    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.224035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.223982    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjpm2\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.225017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.224985    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.225164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.225072    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.230392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.230359    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.246144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.243580    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjpm2\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.251559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.251537    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:17.252901 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:36:17.252873    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58449eb4_5a4f_4ff4_b89f_febb6b475adc.slice/crio-469a99aa61b03451131db3b1ed603ab44ebbf523dbf12b423e79b48116151e85 WatchSource:0}: Error finding container 469a99aa61b03451131db3b1ed603ab44ebbf523dbf12b423e79b48116151e85: Status 404 returned error can't find the container with id 469a99aa61b03451131db3b1ed603ab44ebbf523dbf12b423e79b48116151e85
Oct 30 21:36:17.415837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.399031    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:17.415837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.399604    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:17.475528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.467266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:17.755459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.753365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:17.934892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.934857    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:36:17.963480 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:36:17.961624    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2477b55_6d9a_4457_9e0d_d87e4bbbad76.slice/crio-5bf8f6821c10e3d5e4ffa5af4ddf6a3abea6d8a2c2bcfa799dae0474759028b9 WatchSource:0}: Error finding container 5bf8f6821c10e3d5e4ffa5af4ddf6a3abea6d8a2c2bcfa799dae0474759028b9: Status 404 returned error can't find the container with id 5bf8f6821c10e3d5e4ffa5af4ddf6a3abea6d8a2c2bcfa799dae0474759028b9
Oct 30 21:36:17.995298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.993897    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" event={"ID":"58449eb4-5a4f-4ff4-b89f-febb6b475adc","Type":"ContainerStarted","Data":"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"}
Oct 30 21:36:17.995298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.993946    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" event={"ID":"58449eb4-5a4f-4ff4-b89f-febb6b475adc","Type":"ContainerStarted","Data":"469a99aa61b03451131db3b1ed603ab44ebbf523dbf12b423e79b48116151e85"}
Oct 30 21:36:17.995298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.994171    6738 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" secret="" err="secret \"payload-processing-external-model-e2e--b5c8139c-dockercfg-ch2nk\" not found"
Oct 30 21:36:17.995298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:17.994576    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:18.019829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:18.019770    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" podStartSLOduration=2.019737187 podStartE2EDuration="2.019737187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:18.013229765 +0000 UTC m=+2034.968916993" watchObservedRunningTime="2026-10-30 21:36:18.019737187 +0000 UTC m=+2034.975424417"
Oct 30 21:36:18.087466 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.087424    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-edf96af6b691: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.087466 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.087466    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.087927 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.087521    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:18.587499597 +0000 UTC m=+2035.543186841 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.087927 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.087752    6738 configmap.go:193] Couldn't get configMap ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-plugins-e2e-extmod-a-fb2b58: configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:18.087927 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.087815    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:18.587798102 +0000 UTC m=+2035.543485322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:18.597809 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.597776    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-edf96af6b691: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.597809 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.597804    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.600046 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.597860    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:19.59784376 +0000 UTC m=+2036.553530979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:18.600046 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.598143    6738 configmap.go:193] Couldn't get configMap ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-plugins-e2e-extmod-a-fb2b58: configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:18.600046 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:18.598181    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:19.598171406 +0000 UTC m=+2036.553858611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:18.819388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:18.819362    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:19.004116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.004076    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" event={"ID":"f2477b55-6d9a-4457-9e0d-d87e4bbbad76","Type":"ContainerStarted","Data":"63a7930110f530f59345de3e245b9fd746f337d2e15b074ed21c6da02ce0b709"}
Oct 30 21:36:19.004409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.004382    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:19.004581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.004424    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" event={"ID":"f2477b55-6d9a-4457-9e0d-d87e4bbbad76","Type":"ContainerStarted","Data":"5bf8f6821c10e3d5e4ffa5af4ddf6a3abea6d8a2c2bcfa799dae0474759028b9"}
Oct 30 21:36:19.032577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.032534    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" podStartSLOduration=2.032521894 podStartE2EDuration="2.032521894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:19.028059906 +0000 UTC m=+2035.983747134" watchObservedRunningTime="2026-10-30 21:36:19.032521894 +0000 UTC m=+2035.988209124"
Oct 30 21:36:19.580237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.580208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:19.614459 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:19.613840    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-edf96af6b691: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:19.614459 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:19.613869    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:19.614459 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:19.613926    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:21.613907969 +0000 UTC m=+2038.569595188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:19.614459 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:19.614296    6738 configmap.go:193] Couldn't get configMap ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-plugins-e2e-extmod-a-fb2b58: configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:19.614459 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:19.614344    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:21.614331015 +0000 UTC m=+2038.570018237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:19.701918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:19.701882    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:20.459256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:20.459227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:21.172389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:21.172360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:21.644703 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:21.644673    6738 configmap.go:193] Couldn't get configMap ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-plugins-e2e-extmod-a-fb2b58: configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:21.644884 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:21.644739    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:25.644725366 +0000 UTC m=+2042.600412590 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : configmap "payload-processing-external-model-plugins-e2e-extmod-a-fb2b58" not found
Oct 30 21:36:21.644884 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:21.644764    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-edf96af6b691: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:21.644884 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:21.644779    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn: secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:21.644884 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:21.644836    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials podName:58449eb4-5a4f-4ff4-b89f-febb6b475adc nodeName:}" failed. No retries permitted until 2026-10-30 21:36:25.644812896 +0000 UTC m=+2042.600500105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc") : secret "agc-cred-edf96af6b691" not found
Oct 30 21:36:21.963357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:21.963283    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:22.519153 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:22.519116    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:36:22.701121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:22.701084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:23.380936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:23.380902    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:24.004924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.004878    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" podUID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" containerName="payload-processing" containerID="cri-o://9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a" gracePeriod=25
Oct 30 21:36:24.255829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.255752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:24.287028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.286976    6738 scope.go:122] "RemoveContainer" containerID="160b3bc8644c6c68ca7d81c4f274df3d813def9cf12c5ab2fd141bf37975e8a7"
Oct 30 21:36:24.342897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.342859    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:24.388866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.388814    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\") pod \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") "
Oct 30 21:36:24.389057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.389031    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\") pod \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") "
Oct 30 21:36:24.389160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.389097    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\") pod \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") "
Oct 30 21:36:24.389160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.389153    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\") pod \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\" (UID: \"58449eb4-5a4f-4ff4-b89f-febb6b475adc\") "
Oct 30 21:36:24.390609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.390102    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume" pod "58449eb4-5a4f-4ff4-b89f-febb6b475adc" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:24.390609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.390498    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume" pod "58449eb4-5a4f-4ff4-b89f-febb6b475adc" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:36:24.393360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.393338    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt" pod "58449eb4-5a4f-4ff4-b89f-febb6b475adc" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc"). InnerVolumeSpecName "kube-api-access-s86zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:24.394426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.394399    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials" pod "58449eb4-5a4f-4ff4-b89f-febb6b475adc" (UID: "58449eb4-5a4f-4ff4-b89f-febb6b475adc"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:36:24.491941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.491897    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s86zt\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-kube-api-access-s86zt\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:24.491941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.491933    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:24.492111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.491947    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/58449eb4-5a4f-4ff4-b89f-febb6b475adc-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:24.492111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:24.491961    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58449eb4-5a4f-4ff4-b89f-febb6b475adc-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:36:25.070786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.070749    6738 generic.go:356] "Generic (PLEG): container finished" podID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" containerID="9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a" exitCode=0
Oct 30 21:36:25.071268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.070818    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"
Oct 30 21:36:25.071268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.070834    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" event={"ID":"58449eb4-5a4f-4ff4-b89f-febb6b475adc","Type":"ContainerDied","Data":"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"}
Oct 30 21:36:25.071268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.070911    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn" event={"ID":"58449eb4-5a4f-4ff4-b89f-febb6b475adc","Type":"ContainerDied","Data":"469a99aa61b03451131db3b1ed603ab44ebbf523dbf12b423e79b48116151e85"}
Oct 30 21:36:25.071268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.070934    6738 scope.go:122] "RemoveContainer" containerID="9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"
Oct 30 21:36:25.083096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.083078    6738 scope.go:122] "RemoveContainer" containerID="9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"
Oct 30 21:36:25.083363 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:25.083338    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a\": container with ID starting with 9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a not found: ID does not exist" containerID="9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"
Oct 30 21:36:25.083455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.083369    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a"} err="failed to get container status \"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a\": rpc error: code = NotFound desc = could not find container \"9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a\": container with ID starting with 9af824cc4afa6e91db2022fffe59356ede16419dcf6bd761c17b74fce6f2519a not found: ID does not exist"
Oct 30 21:36:25.099593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.099570    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:25.103580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.103555    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-7767hzngn"]
Oct 30 21:36:25.167194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.167165    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" path="/var/lib/kubelet/pods/58449eb4-5a4f-4ff4-b89f-febb6b475adc/volumes"
Oct 30 21:36:25.235359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:25.235331    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:26.225772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:26.225739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:27.057464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:27.057015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:27.758678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:27.758644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:28.591324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:28.591289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:29.634946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:29.634909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:30.014744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:30.014652    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:36:30.787541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:30.787509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:31.591324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:31.587348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:32.539622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:32.539587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:33.495215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:33.495180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:34.329947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:34.329909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:35.188014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:35.187975    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:36.203802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:36.203209    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:37.116622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:37.116579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:37.824489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:37.824429    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:38.658944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:38.658895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:39.600729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:39.600678    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:40.460059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:40.459983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:41.148735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:41.148697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:42.067228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:42.067195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:42.847371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:42.847336    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:43.790398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:43.790363    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:44.892458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:44.889377    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:45.736931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:45.736892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:46.467334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:46.467290    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:47.288212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:47.288177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:48.088627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:48.088593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:48.991608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:48.991559    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:49.895385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:49.895334    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:50.210998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.210910    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:36:50.215486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.212197    6738 cpu_manager.go:414] "containerMap: removing container" podUID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" containerName="payload-processing"
Oct 30 21:36:50.215486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.212224    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" containerName="payload-processing"
Oct 30 21:36:50.215486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.212429    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="58449eb4-5a4f-4ff4-b89f-febb6b475adc" containerName="payload-processing"
Oct 30 21:36:50.221995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.221968    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.244005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.243981    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:36:50.256731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.256699    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.256946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.256920    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.256999    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257060    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257096    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9wt\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257160    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257233    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257738    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.257883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.257857    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.302449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.302410    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-fb2b58-openshift-default-dockercfg-7jgd8\""
Oct 30 21:36:50.358970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.358942    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359049    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359097    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359132    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359158    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359189    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359236    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359285    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9wt\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359372    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359467    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359629    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359656    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.359762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.359744    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.360292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.360265    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.361748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.361730    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.361958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.361937    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.369464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.369425    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9wt\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.369629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.369609    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.548110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.548044    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:50.861115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.861088    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:36:50.862184 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:36:50.862162    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76a6d50_5d68_437f_87de_34a4edc6ce98.slice/crio-1d45fd02a778009c06c8d092c67f4413804d02b9aa60e1284e78bd3dbd46c7ce WatchSource:0}: Error finding container 1d45fd02a778009c06c8d092c67f4413804d02b9aa60e1284e78bd3dbd46c7ce: Status 404 returned error can't find the container with id 1d45fd02a778009c06c8d092c67f4413804d02b9aa60e1284e78bd3dbd46c7ce
Oct 30 21:36:50.865595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.865556    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:50.865690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.865630    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:50.865690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.865662    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:36:50.983038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:50.983000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:51.353452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.353404    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" event={"ID":"b76a6d50-5d68-437f-87de-34a4edc6ce98","Type":"ContainerStarted","Data":"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"}
Oct 30 21:36:51.353633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.353564    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" event={"ID":"b76a6d50-5d68-437f-87de-34a4edc6ce98","Type":"ContainerStarted","Data":"1d45fd02a778009c06c8d092c67f4413804d02b9aa60e1284e78bd3dbd46c7ce"}
Oct 30 21:36:51.387379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.387326    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podStartSLOduration=1.387312263 podStartE2EDuration="1.387312263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:36:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:36:51.383838019 +0000 UTC m=+2068.339525259" watchObservedRunningTime="2026-10-30 21:36:51.387312263 +0000 UTC m=+2068.342999490"
Oct 30 21:36:51.548448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.548405    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:51.549625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.549592    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:51.549758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.549638    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:51.883460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:51.883218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:52.546410 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:36:52.546380    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:36:52.548518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:52.548485    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:52.548674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:52.548559    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:53.073767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:53.073731    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:53.549375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:53.549338    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:53.549788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:53.549407    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:53.811016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:53.810924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:54.549236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:54.549204    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:54.549494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:54.549469    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:54.563114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:54.563086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:55.483249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:55.483214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:55.549018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:55.548985    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:55.549306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:55.549280    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:56.261095 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:56.261055    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:56.549322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:56.549235    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 21:36:56.549322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:56.549309    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 21:36:57.148196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.148159    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:57.511901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.511815    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:57.651055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.650963    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:57.727662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.721916    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:57.727662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.723607    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:57.735154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:57.735130    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:36:58.411573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:58.411534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:36:59.341410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:36:59.341307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:01.052341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:01.052305    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:01.666281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:01.666247    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:02.370233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:02.370204    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:03.039102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:03.039073    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:37:03.039321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:03.039299    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy" containerID="cri-o://f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980" gracePeriod=30
Oct 30 21:37:03.140327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:03.140293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:03.834044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:03.834008    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:04.574044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:04.573970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:05.181296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.181261    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:37:05.185965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.185947    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.201609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.201583    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:37:05.215502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.215166    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:37:05.223135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.221701    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.243741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.243700    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:37:05.341007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.340969    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.341007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341004    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341270    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341352    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.341564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341545    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvr99\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341682    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.341745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341733    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341773    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341812    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.341944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341932    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.341978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341960    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.342015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.341989    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntdrs\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.342057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342020    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.342092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342078    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.342142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342102    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.342182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342169    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.342230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342201    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.342230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.342219    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.391795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.391756    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:05.443887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.443782    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvr99\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.443911    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.443973    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444030    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444058    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444134    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444170    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444223    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntdrs\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444267    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444316    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444362    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444417    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444627    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444671    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444733    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.444840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444764    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.444840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444817    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.445001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.444891    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.445596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.445567    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.445881    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.446310    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.446423    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.446753    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.446862    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.447097    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.447335    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.448132    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.449352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.448844    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.452641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.452607    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.453054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.452923    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.459599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.453146    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.459599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.454589    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.459599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.458412    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvr99\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.459599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.458815    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.461005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.460980    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.464460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.464417    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntdrs\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.536968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.536935    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:05.548228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.548202    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:05.860401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.859714    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:37:05.860401 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:05.860018    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12dc4ca0_ee6f_48c5_8959_88b05e2f8d5e.slice/crio-4f7044dad978909cf5bfdb9b0e7a5a5f1f750f8b38df51d06c4e53d26a2f2ff5 WatchSource:0}: Error finding container 4f7044dad978909cf5bfdb9b0e7a5a5f1f750f8b38df51d06c4e53d26a2f2ff5: Status 404 returned error can't find the container with id 4f7044dad978909cf5bfdb9b0e7a5a5f1f750f8b38df51d06c4e53d26a2f2ff5
Oct 30 21:37:05.864151 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:05.864126    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b558560_ef9f_4ebc_b90c_3c350fc1bd35.slice/crio-0fd7ebb7820c66bb6052b08d03e1f33ea2161dcdda42e5e288886e6942c3f128 WatchSource:0}: Error finding container 0fd7ebb7820c66bb6052b08d03e1f33ea2161dcdda42e5e288886e6942c3f128: Status 404 returned error can't find the container with id 0fd7ebb7820c66bb6052b08d03e1f33ea2161dcdda42e5e288886e6942c3f128
Oct 30 21:37:05.864497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.864408    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:05.864569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.864514    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:05.864569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.864561    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:05.864947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.864932    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:37:05.867091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.867062    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:05.867198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.867118    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:05.867198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:05.867146    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:06.282573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.282539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:06.517276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.517239    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" event={"ID":"6b558560-ef9f-4ebc-b90c-3c350fc1bd35","Type":"ContainerStarted","Data":"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"}
Oct 30 21:37:06.517603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.517574    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" event={"ID":"6b558560-ef9f-4ebc-b90c-3c350fc1bd35","Type":"ContainerStarted","Data":"0fd7ebb7820c66bb6052b08d03e1f33ea2161dcdda42e5e288886e6942c3f128"}
Oct 30 21:37:06.519389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.519360    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" event={"ID":"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e","Type":"ContainerStarted","Data":"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"}
Oct 30 21:37:06.519525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.519411    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" event={"ID":"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e","Type":"ContainerStarted","Data":"4f7044dad978909cf5bfdb9b0e7a5a5f1f750f8b38df51d06c4e53d26a2f2ff5"}
Oct 30 21:37:06.537959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.537891    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:06.539750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.539691    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:06.539880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.539764    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:06.543289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.543250    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podStartSLOduration=1.5432386710000001 podStartE2EDuration="1.543238671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:06.538687715 +0000 UTC m=+2083.494374956" watchObservedRunningTime="2026-10-30 21:37:06.543238671 +0000 UTC m=+2083.498925899"
Oct 30 21:37:06.548490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.548466    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:06.549883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.549857    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:06.549976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.549908    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:06.564616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:06.564562    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podStartSLOduration=1.564545548 podStartE2EDuration="1.564545548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:06.56347345 +0000 UTC m=+2083.519160688" watchObservedRunningTime="2026-10-30 21:37:06.564545548 +0000 UTC m=+2083.520232778"
Oct 30 21:37:07.266019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:07.265983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:07.548517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:07.548075    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:07.548517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:07.548135    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:07.552749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:07.551026    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:07.552749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:07.551079    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:08.263162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.263126    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:08.478207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.477644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:08.574359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.573562    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:08.574359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.573620    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:08.574359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.573943    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:08.574359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.573987    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:08.684968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.684058    6738 generic.go:356] "Generic (PLEG): container finished" podID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerID="f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980" exitCode=0
Oct 30 21:37:08.684968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.684509    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" event={"ID":"a1871633-1f97-451f-a9c5-8b414001e3a1","Type":"ContainerDied","Data":"f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980"}
Oct 30 21:37:08.684968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.684579    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j" event={"ID":"a1871633-1f97-451f-a9c5-8b414001e3a1","Type":"ContainerDied","Data":"b1e0d5457fcf3e2460a5bd9dcffc35e9bdaefbc427cd58b27a650d8020636aae"}
Oct 30 21:37:08.684968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.684603    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1e0d5457fcf3e2460a5bd9dcffc35e9bdaefbc427cd58b27a650d8020636aae"
Oct 30 21:37:08.702544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.693597    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:37:08.780649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.780423    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.780792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.780727    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.780896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.780869    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:08.780964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.780929    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781095    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781138    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781176    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781244    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781258    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:08.781458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781421    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781470    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\") pod \"a1871633-1f97-451f-a9c5-8b414001e3a1\" (UID: \"a1871633-1f97-451f-a9c5-8b414001e3a1\") "
Oct 30 21:37:08.781580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781519    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:08.781722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.781701    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:08.782083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782055    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:08.782712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782367    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.782712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782392    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.782712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782408    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1871633-1f97-451f-a9c5-8b414001e3a1-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.782712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782422    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.782712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.782453    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.785091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.785057    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:08.785916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.785770    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "kube-api-access-c9xrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:08.785916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.785880    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:08.796468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.794623    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token" pod "a1871633-1f97-451f-a9c5-8b414001e3a1" (UID: "a1871633-1f97-451f-a9c5-8b414001e3a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:08.884149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.884078    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.884149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.884106    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.884149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.884118    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9xrq\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-kube-api-access-c9xrq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:08.884149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:08.884128    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1871633-1f97-451f-a9c5-8b414001e3a1-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:09.313457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.310336    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:09.467510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.460508    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:09.544465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.537788    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:09.544465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.537849    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:09.582241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.570938    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:09.582241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.570996    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:09.661500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.657523    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:09.661500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.658810    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy"
Oct 30 21:37:09.661500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.658834    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy"
Oct 30 21:37:09.661500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.659107    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" containerName="istio-proxy"
Oct 30 21:37:09.825511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.825365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:09.910034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.909920    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"
Oct 30 21:37:09.984089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.983482    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.005354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:09.985231    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.073554    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.073598    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-c8n5j"]
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.027614    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.073713    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.073928    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.098062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.073968    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzw7f\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.262512    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.281931    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.282159    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.282193    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzw7f\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.263613    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.283839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.283514    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.287189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.287156    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.327459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.327153    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzw7f\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.543457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.542462    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:10.543457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.542587    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:10.543457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.543336    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:10.562455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.554909    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:10.562455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:10.554955    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:11.058098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.058062    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:11.140459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.130157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:11.362726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.362691    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1871633-1f97-451f-a9c5-8b414001e3a1" path="/var/lib/kubelet/pods/a1871633-1f97-451f-a9c5-8b414001e3a1/volumes"
Oct 30 21:37:11.540979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.540938    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:11.541305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.541251    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:11.557407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.556742    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:11.557407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.556811    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:11.979963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.979920    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" event={"ID":"0955bf2c-9702-4d50-bb7e-48048d88aead","Type":"ContainerStarted","Data":"cd2ca3db97d7fd61cee10e88048072cdcb7b35b3d74718b1b2bfa0a5b2fabc35"}
Oct 30 21:37:11.980172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.980162    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:11.980252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:11.980190    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" event={"ID":"0955bf2c-9702-4d50-bb7e-48048d88aead","Type":"ContainerStarted","Data":"dee73843b756ff7533fc7c5c561bf9c36c459fe0c4c0aa397cbddf0e28cb89cc"}
Oct 30 21:37:12.059875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.059823    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" podStartSLOduration=3.059809088 podStartE2EDuration="3.059809088s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:12.028614731 +0000 UTC m=+2088.984301959" watchObservedRunningTime="2026-10-30 21:37:12.059809088 +0000 UTC m=+2089.015496315"
Oct 30 21:37:12.136464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.122773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:12.549857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.549821    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused" start-of-body=
Oct 30 21:37:12.550189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.550157    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.133:15021/healthz/ready\": dial tcp 10.134.0.133:15021: connect: connection refused"
Oct 30 21:37:12.551001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.550980    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused" start-of-body=
Oct 30 21:37:12.551165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:12.551146    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.134:15021/healthz/ready\": dial tcp 10.134.0.134:15021: connect: connection refused"
Oct 30 21:37:13.144205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.144173    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:13.543854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.543777    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:13.544274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.544247    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:13.545011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.544988    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:37:13.554068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.554046    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:13.554314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.554292    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:13.555080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.555062    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:37:13.930236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:13.930206    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:14.671566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:14.671528    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:15.366385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:15.366350    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:37:15.366608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:15.366584    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy" containerID="cri-o://948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15" gracePeriod=30
Oct 30 21:37:15.557769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:15.557734    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:16.285119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:16.285088    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:16.926832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:16.926800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:17.767100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:17.767065    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:18.700756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:18.700690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:19.539749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:19.539695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:19.645057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:19.645022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:20.222415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.222378    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:37:20.228424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.228399    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.233718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.233689    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:37:20.241890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.241864    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.242057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.242039    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:37:20.253879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.253843    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:37:20.354083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354037    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.354366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354339    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.354535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354378    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5r2g\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.354599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354574    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.354660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354604    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.354660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354642    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.354775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354740    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.354814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354785    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.354814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354807    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjq55\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.354911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354837    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.354911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354903    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.355013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.354998    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.355055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355022    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.355120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355076    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.355257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355233    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.355332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355312    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.355410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355354    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.355516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.355488    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457264    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457320    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457371    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457407    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457500    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457533    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5r2g\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457592    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457629    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.457887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457668    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457754    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457790    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457818    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjq55\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.457887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457862    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.458158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457900    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.458158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.457932    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.458158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458056    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.458158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458095    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.458372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458188    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.458372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458217    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.458372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458291    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.458686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458665    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.458949    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.459234    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.459858    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.459918    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.460340    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.460550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.460486    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.460951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.460551    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.461984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.461964    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.462092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.461999    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.462092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.462064    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.462167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.462087    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.466599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.466567    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5r2g\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.466772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.466749    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjq55\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.467058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.467039    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.467132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.467051    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.498778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.498750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:20.572581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.572272    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:20.579566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.579289    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:20.759039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.759013    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:37:20.868485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868419    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.868675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868547    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.868675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868587    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.868805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868717    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.868805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868736    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:20.868805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.868746    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.869167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869018    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:20.869167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869056    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.869167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869153    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.870111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869224    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.870111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869287    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\") pod \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\" (UID: \"4eaf560a-6693-42a5-bb53-0fa44d5c79bd\") "
Oct 30 21:37:20.870111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869356    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:20.870111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869516    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:20.870111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.869850    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:20.871027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.870379    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.871027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.870412    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.871027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.870478    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.871027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.870496    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.871027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.870510    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.871922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.871904    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:37:20.872115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.872088    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:37:20.872219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.872202    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:20.873591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.873562    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf" pod "4eaf560a-6693-42a5-bb53-0fa44d5c79bd" (UID: "4eaf560a-6693-42a5-bb53-0fa44d5c79bd"). InnerVolumeSpecName "kube-api-access-hmhbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:20.952320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.952235    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:37:20.952970 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:20.952940    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96c0ef43_3948_49fd_973c_db6859e99e2d.slice/crio-9f95ce67c5846d693f49886eb2403d27094e1a55de5c23e5ec2449b85d32b727 WatchSource:0}: Error finding container 9f95ce67c5846d693f49886eb2403d27094e1a55de5c23e5ec2449b85d32b727: Status 404 returned error can't find the container with id 9f95ce67c5846d693f49886eb2403d27094e1a55de5c23e5ec2449b85d32b727
Oct 30 21:37:20.954141 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:20.954118    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af20973_49cd_4eb8_9b78_035d466cfa18.slice/crio-e1567ee4f96624d9cc906aad7a27dc6b817bba028065b17432a09c156b0902f3 WatchSource:0}: Error finding container e1567ee4f96624d9cc906aad7a27dc6b817bba028065b17432a09c156b0902f3: Status 404 returned error can't find the container with id e1567ee4f96624d9cc906aad7a27dc6b817bba028065b17432a09c156b0902f3
Oct 30 21:37:20.956857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.956808    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.956931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.956914    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.956986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.956956    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.957136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.957111    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:37:20.957552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.957506    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.957617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.957562    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.957617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.957589    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:20.972764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.972734    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.972764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.972762    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.972932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.972778    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:20.972932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:20.972793    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmhbf\" (UniqueName: \"kubernetes.io/projected/4eaf560a-6693-42a5-bb53-0fa44d5c79bd-kube-api-access-hmhbf\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:21.047375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.047351    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:21.099769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.099719    6738 generic.go:356] "Generic (PLEG): container finished" podID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerID="948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15" exitCode=0
Oct 30 21:37:21.099980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.099963    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"
Oct 30 21:37:21.101149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.100649    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" event={"ID":"4eaf560a-6693-42a5-bb53-0fa44d5c79bd","Type":"ContainerDied","Data":"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"}
Oct 30 21:37:21.101149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.100715    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg" event={"ID":"4eaf560a-6693-42a5-bb53-0fa44d5c79bd","Type":"ContainerDied","Data":"24db9435beb754a01a2480f19dcca9daa9da0b2b25ea416e7c9339ba4b7052ab"}
Oct 30 21:37:21.101149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.100748    6738 scope.go:122] "RemoveContainer" containerID="948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"
Oct 30 21:37:21.105166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.105125    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" event={"ID":"96c0ef43-3948-49fd-973c-db6859e99e2d","Type":"ContainerStarted","Data":"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"}
Oct 30 21:37:21.105298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.105190    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" event={"ID":"96c0ef43-3948-49fd-973c-db6859e99e2d","Type":"ContainerStarted","Data":"9f95ce67c5846d693f49886eb2403d27094e1a55de5c23e5ec2449b85d32b727"}
Oct 30 21:37:21.114055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.113124    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" event={"ID":"3af20973-49cd-4eb8-9b78-035d466cfa18","Type":"ContainerStarted","Data":"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"}
Oct 30 21:37:21.114055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.113173    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" event={"ID":"3af20973-49cd-4eb8-9b78-035d466cfa18","Type":"ContainerStarted","Data":"e1567ee4f96624d9cc906aad7a27dc6b817bba028065b17432a09c156b0902f3"}
Oct 30 21:37:21.123379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.123322    6738 scope.go:122] "RemoveContainer" containerID="948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"
Oct 30 21:37:21.123794 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:21.123716    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15\": container with ID starting with 948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15 not found: ID does not exist" containerID="948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"
Oct 30 21:37:21.123794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.123764    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15"} err="failed to get container status \"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15\": rpc error: code = NotFound desc = could not find container \"948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15\": container with ID starting with 948b1ba384e84a4fc59a4f91d7e6fd0f966ebfba449ebbd260f7ab28b3d81c15 not found: ID does not exist"
Oct 30 21:37:21.129479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.129395    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podStartSLOduration=1.129378755 podStartE2EDuration="1.129378755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:21.126049232 +0000 UTC m=+2098.081736450" watchObservedRunningTime="2026-10-30 21:37:21.129378755 +0000 UTC m=+2098.085065984"
Oct 30 21:37:21.150769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.150748    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:37:21.179578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.177971    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-cjmrg"]
Oct 30 21:37:21.350492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.350397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:21.579043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.578741    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:21.579643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.579587    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:21.615244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.607350    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:21.741588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:21.737916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:22.346462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.344991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:22.477469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.477326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:22.612779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.597100    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:22.612779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.597169    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:22.613908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.613753    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:22.613908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.613817    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:22.636734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.636666    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podStartSLOduration=2.636644028 podStartE2EDuration="2.636644028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:21.174939619 +0000 UTC m=+2098.130626847" watchObservedRunningTime="2026-10-30 21:37:22.636644028 +0000 UTC m=+2099.592331257"
Oct 30 21:37:22.697003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.686519    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:37:22.748790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.739036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:22.886347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.883268    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:22.894465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:22.888423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:23.092251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.092203    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:23.117478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.115901    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:23.569957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.569663    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:23.586670 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.586628    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:23.766777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.587943    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Oct 30 21:37:23.814781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.622687    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:23.815902 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:23.624453    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:37:23.816100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.782691    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.783336    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.978494    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.959901    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.978552    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:23.800910    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.034473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.001164    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.383491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.382031    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" path="/var/lib/kubelet/pods/4eaf560a-6693-42a5-bb53-0fa44d5c79bd/volumes"
Oct 30 21:37:24.383491 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:24.382538    6738 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.131s"
Oct 30 21:37:24.409036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.409002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:24.448489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.447856    6738 scope.go:122] "RemoveContainer" containerID="f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980"
Oct 30 21:37:24.585819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.585497    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:24.585819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.585555    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:24.621634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.587712    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:24.621634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.587776    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:24.661878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.631029    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.661878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.631470    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:37:24.757228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.757156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:24.830462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.820745    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.830462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.821210    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:37:24.870415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.835518    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 21:37:24.870415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.835943    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:37:24.870415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.837801    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:24.870415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.838140    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:37:25.007514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:24.979452    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:37:25.041615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.037467    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:37:25.041615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.014468    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:25.041615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.038168    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:37:25.440498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.440467    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:37:25.574583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.574553    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:25.574794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.574608    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:25.581517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.581026    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:25.581517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.581073    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:25.665459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.665064    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:25.750512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.749706    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:37:25.767662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.767632    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:37:25.865149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.865110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:25.874012 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:25.873977    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980\": container with ID starting with f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980 not found: ID does not exist" containerID="f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980"
Oct 30 21:37:25.874226 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:25.874205    6738 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="get container status: runtime container status: rpc error: code = NotFound desc = could not find container \"f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980\": container with ID starting with f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980 not found: ID does not exist"
Oct 30 21:37:25.879466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:25.879423    6738 image_gc_manager.go:223] "Failed to monitor images" err="get container status: runtime container status: rpc error: code = NotFound desc = could not find container \"f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980\": container with ID starting with f6306b91f17d75197619617f9ece28f0cb902fbdf8303cf6cafae3bd2ea55980 not found: ID does not exist"
Oct 30 21:37:26.211850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.211781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.379091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.379019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.554507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.552466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:26.574746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.574619    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:26.574933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.574772    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:26.582478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.580218    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:26.582478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.580289    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:26.714944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:26.714909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:27.276461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.266409    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:27.419459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.415373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:27.579518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.578578    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:27.579518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.578641    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:27.579781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.579753    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:27.579838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.579784    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:27.707634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.687511    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" podUID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" containerName="payload-processing" containerID="cri-o://63a7930110f530f59345de3e245b9fd746f337d2e15b074ed21c6da02ce0b709" gracePeriod=25
Oct 30 21:37:27.893458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.885505    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" podUID="0955bf2c-9702-4d50-bb7e-48048d88aead" containerName="payload-processing" containerID="cri-o://cd2ca3db97d7fd61cee10e88048072cdcb7b35b3d74718b1b2bfa0a5b2fabc35" gracePeriod=25
Oct 30 21:37:28.000485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.980387    6738 generic.go:356] "Generic (PLEG): container finished" podID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" containerID="63a7930110f530f59345de3e245b9fd746f337d2e15b074ed21c6da02ce0b709" exitCode=0
Oct 30 21:37:28.000485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:27.981834    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" event={"ID":"f2477b55-6d9a-4457-9e0d-d87e4bbbad76","Type":"ContainerDied","Data":"63a7930110f530f59345de3e245b9fd746f337d2e15b074ed21c6da02ce0b709"}
Oct 30 21:37:28.055667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.042940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:28.168247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.168215    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:28.582338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.578253    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:28.582338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.578310    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:28.582338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.580743    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:28.582338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.580783    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:28.790511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.777493    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:37:28.817016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.810650    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy"
Oct 30 21:37:28.817016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.810770    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy"
Oct 30 21:37:28.817016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.811423    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4eaf560a-6693-42a5-bb53-0fa44d5c79bd" containerName="istio-proxy"
Oct 30 21:37:28.888674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.877370    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:28.899484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.895086    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.944095    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"agc-cred-2036742396f6\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.944258    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"kube-root-ca.crt\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.944922    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"payload-processing-external-model-plugins-e2e-extmod-b-36a099\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.945005    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"routing-overlay\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.945062    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"openshift-service-ca.crt\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.945118    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"agc-cred-2f614df36ac6\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.945227    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"payload-processing-external-model-e2e--3e617b48-dockercfg-d4sw2\""
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.920647    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg9ng\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.959894    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.960172    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:28.963301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.960383    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:28.976469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:28.973649    6738 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/health\": context deadline exceeded"
Oct 30 21:37:29.028495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.024822    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:37:29.048463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.047275    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:37:29.076457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.073765    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.076457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.075822    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.076457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.075967    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9ng\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.076457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.076006    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.082130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.077052    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.082130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.077580    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.131659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.131625    6738 generic.go:356] "Generic (PLEG): container finished" podID="0955bf2c-9702-4d50-bb7e-48048d88aead" containerID="cd2ca3db97d7fd61cee10e88048072cdcb7b35b3d74718b1b2bfa0a5b2fabc35" exitCode=0
Oct 30 21:37:29.132014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.131989    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" event={"ID":"0955bf2c-9702-4d50-bb7e-48048d88aead","Type":"ContainerDied","Data":"cd2ca3db97d7fd61cee10e88048072cdcb7b35b3d74718b1b2bfa0a5b2fabc35"}
Oct 30 21:37:29.132167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.132151    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt" event={"ID":"0955bf2c-9702-4d50-bb7e-48048d88aead","Type":"ContainerDied","Data":"dee73843b756ff7533fc7c5c561bf9c36c459fe0c4c0aa397cbddf0e28cb89cc"}
Oct 30 21:37:29.132285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.132273    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dee73843b756ff7533fc7c5c561bf9c36c459fe0c4c0aa397cbddf0e28cb89cc"
Oct 30 21:37:29.156759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.133915    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9ng\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.156759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.136620    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:29.181980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.180859    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.181980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.181881    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\") pod \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") "
Oct 30 21:37:29.183709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.182549    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\") pod \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") "
Oct 30 21:37:29.183709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.182624    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\") pod \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") "
Oct 30 21:37:29.183709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.182837    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\") pod \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\" (UID: \"f2477b55-6d9a-4457-9e0d-d87e4bbbad76\") "
Oct 30 21:37:29.183709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.182922    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume" pod "f2477b55-6d9a-4457-9e0d-d87e4bbbad76" (UID: "f2477b55-6d9a-4457-9e0d-d87e4bbbad76"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:29.183709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.183290    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume" pod "f2477b55-6d9a-4457-9e0d-d87e4bbbad76" (UID: "f2477b55-6d9a-4457-9e0d-d87e4bbbad76"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:29.184139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.183826    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.184139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.183850    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.261307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.202057    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"
Oct 30 21:37:29.262965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.261910    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials" pod "f2477b55-6d9a-4457-9e0d-d87e4bbbad76" (UID: "f2477b55-6d9a-4457-9e0d-d87e4bbbad76"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:29.266521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.266501    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:29.267543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.266949    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2" pod "f2477b55-6d9a-4457-9e0d-d87e4bbbad76" (UID: "f2477b55-6d9a-4457-9e0d-d87e4bbbad76"). InnerVolumeSpecName "kube-api-access-kjpm2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:29.278119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.278093    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g" event={"ID":"f2477b55-6d9a-4457-9e0d-d87e4bbbad76","Type":"ContainerDied","Data":"5bf8f6821c10e3d5e4ffa5af4ddf6a3abea6d8a2c2bcfa799dae0474759028b9"}
Oct 30 21:37:29.278200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.278178    6738 scope.go:122] "RemoveContainer" containerID="63a7930110f530f59345de3e245b9fd746f337d2e15b074ed21c6da02ce0b709"
Oct 30 21:37:29.284994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.284893    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\") pod \"0955bf2c-9702-4d50-bb7e-48048d88aead\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") "
Oct 30 21:37:29.285141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.285016    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\") pod \"0955bf2c-9702-4d50-bb7e-48048d88aead\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") "
Oct 30 21:37:29.285141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.285117    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\") pod \"0955bf2c-9702-4d50-bb7e-48048d88aead\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") "
Oct 30 21:37:29.285258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.285240    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\") pod \"0955bf2c-9702-4d50-bb7e-48048d88aead\" (UID: \"0955bf2c-9702-4d50-bb7e-48048d88aead\") "
Oct 30 21:37:29.286326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.285969    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume" pod "0955bf2c-9702-4d50-bb7e-48048d88aead" (UID: "0955bf2c-9702-4d50-bb7e-48048d88aead"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:29.286326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.286287    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjpm2\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-kube-api-access-kjpm2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.286326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.286301    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f2477b55-6d9a-4457-9e0d-d87e4bbbad76-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.286326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.286310    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.286811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.286747    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume" pod "0955bf2c-9702-4d50-bb7e-48048d88aead" (UID: "0955bf2c-9702-4d50-bb7e-48048d88aead"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:37:29.351194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.351097    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f" pod "0955bf2c-9702-4d50-bb7e-48048d88aead" (UID: "0955bf2c-9702-4d50-bb7e-48048d88aead"). InnerVolumeSpecName "kube-api-access-wzw7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:29.362246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.354355    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials" pod "0955bf2c-9702-4d50-bb7e-48048d88aead" (UID: "0955bf2c-9702-4d50-bb7e-48048d88aead"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:37:29.444460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.438419    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzw7f\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-kube-api-access-wzw7f\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.444460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.438476    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/0955bf2c-9702-4d50-bb7e-48048d88aead-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.444460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.438494    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/0955bf2c-9702-4d50-bb7e-48048d88aead-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:37:29.585334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.575110    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused" start-of-body=
Oct 30 21:37:29.585334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.575173    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.136:15021/healthz/ready\": dial tcp 10.134.0.136:15021: connect: connection refused"
Oct 30 21:37:29.586027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.585982    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 21:37:29.586138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.586036    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 21:37:29.602518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.589304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:29.687276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.684777    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:37:29.706393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.706242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:29.736459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.730203    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-f9b9xhj7g"]
Oct 30 21:37:29.836141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.836049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:29.930912 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:29.930875    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod235875c7_e9ff_4fec_a4b9_df3d4cd0389a.slice/crio-b4e08657ab4ebccdd17396b670ce3077d5e7cf9270235b3d613ea0dba28b4eec WatchSource:0}: Error finding container b4e08657ab4ebccdd17396b670ce3077d5e7cf9270235b3d613ea0dba28b4eec: Status 404 returned error can't find the container with id b4e08657ab4ebccdd17396b670ce3077d5e7cf9270235b3d613ea0dba28b4eec
Oct 30 21:37:29.936548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.932201    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:37:29.963171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:29.963141    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:30.213028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.212925    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" event={"ID":"235875c7-e9ff-4fec-a4b9-df3d4cd0389a","Type":"ContainerStarted","Data":"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"}
Oct 30 21:37:30.213028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.212983    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" event={"ID":"235875c7-e9ff-4fec-a4b9-df3d4cd0389a","Type":"ContainerStarted","Data":"b4e08657ab4ebccdd17396b670ce3077d5e7cf9270235b3d613ea0dba28b4eec"}
Oct 30 21:37:30.213028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.213012    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:30.213333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.213037    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"
Oct 30 21:37:30.232802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.232746    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" podStartSLOduration=2.232729798 podStartE2EDuration="2.232729798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:30.230553961 +0000 UTC m=+2107.186241184" watchObservedRunningTime="2026-10-30 21:37:30.232729798 +0000 UTC m=+2107.188417028"
Oct 30 21:37:30.266966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.266935    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:30.274270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.274239    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-7897xpcwt"]
Oct 30 21:37:30.579356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.579275    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:30.579749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.579713    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:30.580555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.580534    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:37:30.585997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.585972    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:30.586421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.586264    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:30.586970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.586954    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:37:30.827296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:30.827260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:31.166626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:31.166596    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0955bf2c-9702-4d50-bb7e-48048d88aead" path="/var/lib/kubelet/pods/0955bf2c-9702-4d50-bb7e-48048d88aead/volumes"
Oct 30 21:37:31.167205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:31.167183    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" path="/var/lib/kubelet/pods/f2477b55-6d9a-4457-9e0d-d87e4bbbad76/volumes"
Oct 30 21:37:31.733355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:31.733320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.401481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:32.401398    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.582459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:32.560503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.696121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:32.696039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:32.964532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:32.964364    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.123261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:33.123225    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.286203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:33.286115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.443953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:33.443920    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.600521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:33.600402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:33.841011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:33.840978    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.349903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:34.349865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.494009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:34.493977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:34.890250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:34.890207    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.437719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:35.437678    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:35.600954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:35.600906    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.283991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.283949    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:37:36.284876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.284859    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" containerName="payload-processing"
Oct 30 21:37:36.284876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.284877    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" containerName="payload-processing"
Oct 30 21:37:36.285015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.284927    6738 cpu_manager.go:414] "containerMap: removing container" podUID="0955bf2c-9702-4d50-bb7e-48048d88aead" containerName="payload-processing"
Oct 30 21:37:36.285015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.284932    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0955bf2c-9702-4d50-bb7e-48048d88aead" containerName="payload-processing"
Oct 30 21:37:36.285077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.285065    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2477b55-6d9a-4457-9e0d-d87e4bbbad76" containerName="payload-processing"
Oct 30 21:37:36.285110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.285081    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="0955bf2c-9702-4d50-bb7e-48048d88aead" containerName="payload-processing"
Oct 30 21:37:36.303009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.302979    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:37:36.303144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.303087    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317263    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"openshift-service-ca.crt\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317354    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"agc-cred-4af67f87aa6b\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317409    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"agc-cred-b22596d4938d\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317523    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"payload-processing-external-model-e2e--42f38b37-dockercfg-6wsvt\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317599    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"kube-root-ca.crt\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317654    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"routing-overlay\""
Oct 30 21:37:36.319836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.317722    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-729802\"/\"payload-processing-external-model-plugins-e2e-extmod-a-729802\""
Oct 30 21:37:36.361121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.361085    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.361358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.361333    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.361470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.361407    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x2kh\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.361552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.361523    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.463639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.463606    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.464031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.463761    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.464031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.463779    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x2kh\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.464031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.463807    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.464429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.464410    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.464795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.464775    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.466297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.466265    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.473901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.473876    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x2kh\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\") pod \"payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") " pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.624650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.624619    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:36.813342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.813251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:36.913248 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:36.913215    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc90f9c4_89a7_4b55_ada9_55c16dfad309.slice/crio-c0fbf961d4e029d55ab26fd154b707ed4890994f5b220c2b56849b2e3be19b41 WatchSource:0}: Error finding container c0fbf961d4e029d55ab26fd154b707ed4890994f5b220c2b56849b2e3be19b41: Status 404 returned error can't find the container with id c0fbf961d4e029d55ab26fd154b707ed4890994f5b220c2b56849b2e3be19b41
Oct 30 21:37:36.914479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:36.914452    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:37:37.276812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.276709    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" event={"ID":"bc90f9c4-89a7-4b55-ada9-55c16dfad309","Type":"ContainerStarted","Data":"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"}
Oct 30 21:37:37.276812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.276779    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" event={"ID":"bc90f9c4-89a7-4b55-ada9-55c16dfad309","Type":"ContainerStarted","Data":"c0fbf961d4e029d55ab26fd154b707ed4890994f5b220c2b56849b2e3be19b41"}
Oct 30 21:37:37.277022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.276857    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:37.307771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.307710    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" podStartSLOduration=1.307697045 podStartE2EDuration="1.307697045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:37.30598387 +0000 UTC m=+2114.261671097" watchObservedRunningTime="2026-10-30 21:37:37.307697045 +0000 UTC m=+2114.263384253"
Oct 30 21:37:37.321406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.321374    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.477974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.477937    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.679049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.679013    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:37.852460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:37.848854    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:38.010514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:38.004006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:38.796885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:38.796853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:39.129457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:39.126054    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:39.311349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:39.307183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:39.479511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:39.471161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:41.115458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:41.115307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:41.258462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:41.254609    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:37:41.494508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:41.489189    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:42.456408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:42.456192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:43.233906 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:43.233864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:44.165782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.165740    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:44.829406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.829369    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:37:44.834862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.834838    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.846791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.846765    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-729802-openshift-default-dockercfg-4wqzl\""
Oct 30 21:37:44.853372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.850319    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:37:44.940968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.940942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:44.953376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953344    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlj9p\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953516    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953553    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953585    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953621    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953780    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953869    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953946    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:44.953999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:44.953990    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055342    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055378    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055404    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055557    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055631    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055716    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055836    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.055893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055844    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055927    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.055927    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.056082    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.056118    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlj9p\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.056176    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.056301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.056230    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.058356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.058331    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.058585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.058565    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.064873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.064845    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.066129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.066057    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlj9p\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\") pod \"e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") " pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.166078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.166052    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:45.470455 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:37:45.470409    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d437ef5_cade_4b2b_9631_29fdb0036a18.slice/crio-d406082de20844f37b91be2565a538e5113525111cbd954cae1874aa516adf97 WatchSource:0}: Error finding container d406082de20844f37b91be2565a538e5113525111cbd954cae1874aa516adf97: Status 404 returned error can't find the container with id d406082de20844f37b91be2565a538e5113525111cbd954cae1874aa516adf97
Oct 30 21:37:45.471475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.471429    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:37:45.473413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.473377    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:45.473526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.473504    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:45.473567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.473557    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:37:45.486759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.486736    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" event={"ID":"8d437ef5-cade-4b2b-9631-29fdb0036a18","Type":"ContainerStarted","Data":"d406082de20844f37b91be2565a538e5113525111cbd954cae1874aa516adf97"}
Oct 30 21:37:45.853616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:45.853523    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:46.561129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:46.561087    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" event={"ID":"8d437ef5-cade-4b2b-9631-29fdb0036a18","Type":"ContainerStarted","Data":"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"}
Oct 30 21:37:46.562253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:46.562201    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:46.607255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:46.607184    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podStartSLOduration=2.607160487 podStartE2EDuration="2.607160487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:37:46.597813734 +0000 UTC m=+2123.553500962" watchObservedRunningTime="2026-10-30 21:37:46.607160487 +0000 UTC m=+2123.562847714"
Oct 30 21:37:47.168151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.168115    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:47.169327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.169296    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:37:47.169501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.169353    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:37:47.434080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.433983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.642783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.642739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:47.823800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:47.823649    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:48.166428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:48.166389    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:37:48.166617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:48.166467    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:37:48.336242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:48.336193    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:37:48.381457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:48.381411    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:49.166873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:49.166841    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:37:49.167350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:49.166927    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:37:49.432032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:49.431963    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:50.166677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:50.166644    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:37:50.166869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:50.166723    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:37:50.352115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:50.352071    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.112129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:51.112062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:51.170891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:51.170846    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 21:37:51.171204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:51.171174    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 21:37:51.956519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:51.956460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:52.171995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:52.171966    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:52.172474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:52.172445    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:52.173301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:52.173280    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:37:52.515812 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:37:52.515774    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:37:52.657336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:52.657305    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:53.325620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:53.325571    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:54.083348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:54.083317    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:54.816843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:54.816797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:55.669641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:55.669597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:56.359736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:56.359661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:57.130297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:57.130246    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:57.875605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:57.871738    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:58.774007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:58.773973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:37:59.818517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:37:59.818481    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:00.662605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:00.662562    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:02.094405 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:02.094355    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:03.104023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:03.103991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:03.843185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:03.843152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:04.750266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:04.750226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.161357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:05.161174    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.293199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:05.293166    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:05.445514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:05.439407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:06.213307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:06.213275    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:07.244551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:07.244519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.060945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:08.060910    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:08.830110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:08.830065    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:09.546739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:09.546702    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:10.238485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:10.238418    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.023540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:11.023488    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:11.812741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:11.812712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:12.519333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:12.519302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:13.186811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:13.186768    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:13.956421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:13.956383    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:14.761487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:14.761452    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:15.633595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:15.633547    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:16.393151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:16.393083    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:17.261779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:17.261689    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:18.209635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:18.209601    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.047085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:19.047051    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.205986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:19.205946    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:38:19.663206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:19.663178    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:19.955321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:19.955239    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:20.651867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:20.651835    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:21.319965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:21.319932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:22.123964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:22.123908    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:22.522322 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:38:22.522235    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:38:22.589166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:22.589134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:22.702884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:22.702854    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:23.435147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:23.435110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.182999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.182961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:24.219544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.219501    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" podUID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" containerName="payload-processing" containerID="cri-o://87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da" gracePeriod=25
Oct 30 21:38:24.535915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.535890    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:38:24.723983 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.723940    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\") pod \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") "
Oct 30 21:38:24.724187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.724031    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\") pod \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") "
Oct 30 21:38:24.724187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.724109    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\") pod \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") "
Oct 30 21:38:24.724187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.724142    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\") pod \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\" (UID: \"235875c7-e9ff-4fec-a4b9-df3d4cd0389a\") "
Oct 30 21:38:24.724545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.724511    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume" pod "235875c7-e9ff-4fec-a4b9-df3d4cd0389a" (UID: "235875c7-e9ff-4fec-a4b9-df3d4cd0389a"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:24.724643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.724559    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume" pod "235875c7-e9ff-4fec-a4b9-df3d4cd0389a" (UID: "235875c7-e9ff-4fec-a4b9-df3d4cd0389a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:38:24.725135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.725112    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:24.725226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.725140    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:24.726541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.726517    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials" pod "235875c7-e9ff-4fec-a4b9-df3d4cd0389a" (UID: "235875c7-e9ff-4fec-a4b9-df3d4cd0389a"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:24.726665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.726634    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng" pod "235875c7-e9ff-4fec-a4b9-df3d4cd0389a" (UID: "235875c7-e9ff-4fec-a4b9-df3d4cd0389a"). InnerVolumeSpecName "kube-api-access-hg9ng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:38:24.826228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.826157    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg9ng\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-kube-api-access-hg9ng\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:24.826228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.826181    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/235875c7-e9ff-4fec-a4b9-df3d4cd0389a-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:38:24.942659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:24.942631    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:25.075923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.075890    6738 generic.go:356] "Generic (PLEG): container finished" podID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" containerID="87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da" exitCode=0
Oct 30 21:38:25.076115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.075943    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"
Oct 30 21:38:25.076115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.075970    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" event={"ID":"235875c7-e9ff-4fec-a4b9-df3d4cd0389a","Type":"ContainerDied","Data":"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"}
Oct 30 21:38:25.076115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.076024    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw" event={"ID":"235875c7-e9ff-4fec-a4b9-df3d4cd0389a","Type":"ContainerDied","Data":"b4e08657ab4ebccdd17396b670ce3077d5e7cf9270235b3d613ea0dba28b4eec"}
Oct 30 21:38:25.076115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.076045    6738 scope.go:122] "RemoveContainer" containerID="87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"
Oct 30 21:38:25.087766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.087750    6738 scope.go:122] "RemoveContainer" containerID="87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"
Oct 30 21:38:25.088090 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:38:25.088060    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da\": container with ID starting with 87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da not found: ID does not exist" containerID="87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"
Oct 30 21:38:25.088204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.088094    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da"} err="failed to get container status \"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da\": rpc error: code = NotFound desc = could not find container \"87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da\": container with ID starting with 87a9801a145bc65b9095eb3da7951eb6d9adc9ae03d5728cc486762c16bf78da not found: ID does not exist"
Oct 30 21:38:25.101637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.101615    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:38:25.110466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.110423    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-6689rc5vw"]
Oct 30 21:38:25.167076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.167051    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" path="/var/lib/kubelet/pods/235875c7-e9ff-4fec-a4b9-df3d4cd0389a/volumes"
Oct 30 21:38:25.652810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:25.652778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:26.321869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:26.321818    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:26.990804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:26.990769    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:27.669625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:27.669587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:28.349567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:28.349537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:29.402569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:29.402465    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:30.262931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:30.262896    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:31.009337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:31.009296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:31.126499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:31.126454    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:31.905134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:31.905090    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:32.803770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:32.803313    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:33.727340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:33.727295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:34.310098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:34.310026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:34.434179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:34.434094    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:34.912692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:34.909946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:35.330486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:35.330367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:36.100726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:36.100688    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:36.938131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:36.938099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:37.752973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:37.752932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:38.437734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:38.437686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:39.156988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:39.156955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:39.926586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:39.926550    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:40.647397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:40.647365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.338714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:41.338679    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.463815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:41.463784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.581773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:41.581731    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:41.685272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:41.685239    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:42.058686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:42.058605    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:42.741286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:42.741252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.655460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:43.653712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.858968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:43.858937    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:43.985876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:43.978863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:45.149279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:45.149240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:45.863555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:45.863520    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:46.546795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:46.546762    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.346286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:47.346246    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.624871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:47.624824    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:47.918424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:47.918312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.085881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:48.085846    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.241148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:48.240992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.384594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:48.384563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.532610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:48.532533    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:48.687506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:48.687420    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:49.176102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:49.176065    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:49.416974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:49.416940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:49.860600 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:49.860567    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.080506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:50.080422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:50.217845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:50.217816    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:52.008463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:52.002551    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:38:52.008463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:52.003016    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:38:52.237336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:52.237298    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:52.513888 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:38:52.513824    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:38:53.170657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:53.170621    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:53.959971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:53.959919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:55.076002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:55.075971    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:55.270519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:55.270481    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.306735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:56.306703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.444512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:56.444470    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:56.896632 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:56.896601    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:57.047450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:57.047387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:57.831864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:57.831820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:58.844317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:58.844273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:38:59.566428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:38:59.566393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:00.351741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:00.351692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.187533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:01.187499    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.590781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:01.590658    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:01.716042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:01.715991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:02.196481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:02.196401    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:02.919109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:02.919074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:03.029027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:03.028998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:03.647891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:03.647852    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:04.451366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:04.451335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:05.228322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:05.228283    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:06.034162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:06.034121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:06.751225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:06.751183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:07.551546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:07.551407    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:08.324565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:08.324524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:08.856458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:08.856063    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:08.977332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:08.977296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.108736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:09.108644    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.507498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:09.501832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:09.686224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:09.686194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.009075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:10.009044    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.178936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:10.178904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.312042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:10.311945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.728131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:10.728099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:10.857981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:10.857909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:11.498934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:11.498901    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:12.143979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:12.143941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:12.262245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:12.262196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:12.520511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:12.519723    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:12.895743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:12.895691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.634362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:13.634323    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:13.788420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:13.788384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:14.342069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:14.342040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:14.568411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:14.568369    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:15.559900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:15.559838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:16.061072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:16.061018    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:16.190039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:16.189928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:16.995558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:16.995523    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.140260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:17.140221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.453597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:17.453565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.622305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:17.622271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:17.785644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:17.783387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:18.066598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:18.066485    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:19.066271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:19.066212    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:19.831195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:19.831161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:19.959360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:19.959332    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:20.543611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:20.543578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.246063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:21.246034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:21.888306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:21.888266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:22.513195 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:39:22.513164    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:39:22.628086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:22.628046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:23.249244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:23.249213    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:23.423449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:23.423393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.190953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:24.190927    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.317710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:24.317673    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:24.964569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:24.964502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:25.174201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:25.174166    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:25.293520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:25.293180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:26.286731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:26.286696    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:27.145048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:27.145015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:27.839625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:27.839591    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:29.126480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:29.126449    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:30.764414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:30.764378    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:31.738598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:31.738529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:32.753318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:32.753278    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:33.852168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:33.852127    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:34.754638 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:34.754608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:35.601258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:35.601223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:36.303625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:36.303588    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:37.048285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:37.048254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:37.737662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:37.737624    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:38.505280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:38.505250    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:39.357609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:39.357579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:40.111062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:40.111026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:40.861178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:40.861146    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:41.662576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:41.662543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:42.440494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:42.440462    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:43.069425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:43.069392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:43.782891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:43.782859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:44.684087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:44.684040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:45.547607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:45.547570    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:46.314819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:46.314779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:47.074901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.074865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:47.335672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.335597    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:39:47.339224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.338656    6738 cpu_manager.go:414] "containerMap: removing container" podUID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" containerName="payload-processing"
Oct 30 21:39:47.339224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.338681    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" containerName="payload-processing"
Oct 30 21:39:47.339224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.338939    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="235875c7-e9ff-4fec-a4b9-df3d4cd0389a" containerName="payload-processing"
Oct 30 21:39:47.343189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.343167    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:39:47.343508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.343421    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.381804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.381771    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.381927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.381899    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.381969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.381944    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trgmx\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.381984    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.382101    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.382207    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.382266    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.382347    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.382396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.382385    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483252    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483296    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trgmx\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483333    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483371    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483448    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483511    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483575    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483628    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483703    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.483986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.483839    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.484136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.484113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.484201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.484136    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.484201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.484152    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.484201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.484173    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.485943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.485914    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.486246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.486229    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.492181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.492156    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.492322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.492304    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trgmx\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\") pod \"maas-default-gateway-openshift-default-75bf867b6c-46k27\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") " pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.663911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.663873    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:47.930310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:47.930229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:48.010809 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:39:48.010767    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc6e654_4219_4f28_a8df_57f51f09a730.slice/crio-e25627793ff70f6954a9e654a6500feefd61dda4a48aee50a34bc54a16132263 WatchSource:0}: Error finding container e25627793ff70f6954a9e654a6500feefd61dda4a48aee50a34bc54a16132263: Status 404 returned error can't find the container with id e25627793ff70f6954a9e654a6500feefd61dda4a48aee50a34bc54a16132263
Oct 30 21:39:48.016471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.015662    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:48.016471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.015838    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:48.016471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.015924    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:39:48.043052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.043021    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:39:48.582349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.582306    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" event={"ID":"2bc6e654-4219-4f28-a8df-57f51f09a730","Type":"ContainerStarted","Data":"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"}
Oct 30 21:39:48.582786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.582408    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" event={"ID":"2bc6e654-4219-4f28-a8df-57f51f09a730","Type":"ContainerStarted","Data":"e25627793ff70f6954a9e654a6500feefd61dda4a48aee50a34bc54a16132263"}
Oct 30 21:39:48.608002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.607944    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podStartSLOduration=1.607925738 podStartE2EDuration="1.607925738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:39:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:39:48.601152238 +0000 UTC m=+2245.556839463" watchObservedRunningTime="2026-10-30 21:39:48.607925738 +0000 UTC m=+2245.563612967"
Oct 30 21:39:48.664947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.664917    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:48.727931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:48.727848    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:49.533940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:49.533907    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:49.667306 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:49.667279    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:49.667760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:49.667319    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:50.454634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:50.454600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:50.664685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:50.664648    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:50.664875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:50.664698    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:51.186359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:51.186328    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:51.664307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:51.664271    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:51.664521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:51.664335    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:51.954457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:51.954357    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:52.534810 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:39:52.534770    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:39:52.664734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:52.664697    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:52.664923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:52.664773    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:52.873072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:52.873039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:53.657072 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:53.657041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:53.664649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:53.664618    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:53.664806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:53.664667    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:54.433977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:54.433939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:54.665142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:54.665104    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:54.665661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:54.665160    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:55.248261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:55.248221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:55.664409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:55.664374    6738 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-75bf867b6c-46k27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 21:39:55.664617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:55.664457    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 21:39:56.049054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:56.048960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:56.857917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:56.857883    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:56.991426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:56.991387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:57.292208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:57.292178    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:57.293517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:57.293488    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:57.294775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:57.294739    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:39:57.912422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:57.912394    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:39:57.947602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:57.946754    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:58.058041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:58.058004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:58.545461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:58.545046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:58.665114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:58.665077    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.228843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:59.228797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.367090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:59.367023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:39:59.954661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:39:59.954630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:00.657087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:00.657056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:01.476896 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:01.476862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:02.320991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:02.320959    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:02.952231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:02.952178    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" podUID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" containerName="payload-processing" containerID="cri-o://a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b" gracePeriod=25
Oct 30 21:40:03.085496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.083746    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:03.331051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.331016    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:40:03.401399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.401370    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\") pod \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") "
Oct 30 21:40:03.402701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.401502    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\") pod \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") "
Oct 30 21:40:03.402701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.401566    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\") pod \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") "
Oct 30 21:40:03.402701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.402086    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\") pod \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\" (UID: \"bc90f9c4-89a7-4b55-ada9-55c16dfad309\") "
Oct 30 21:40:03.402701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.402235    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume" pod "bc90f9c4-89a7-4b55-ada9-55c16dfad309" (UID: "bc90f9c4-89a7-4b55-ada9-55c16dfad309"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:03.403127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.402946    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:40:03.403330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.403238    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume" pod "bc90f9c4-89a7-4b55-ada9-55c16dfad309" (UID: "bc90f9c4-89a7-4b55-ada9-55c16dfad309"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:40:03.405424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.405401    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials" pod "bc90f9c4-89a7-4b55-ada9-55c16dfad309" (UID: "bc90f9c4-89a7-4b55-ada9-55c16dfad309"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:03.406330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.406303    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh" pod "bc90f9c4-89a7-4b55-ada9-55c16dfad309" (UID: "bc90f9c4-89a7-4b55-ada9-55c16dfad309"). InnerVolumeSpecName "kube-api-access-6x2kh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:40:03.504884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.504856    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/bc90f9c4-89a7-4b55-ada9-55c16dfad309-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:40:03.504884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.504885    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6x2kh\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-kube-api-access-6x2kh\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:40:03.505083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.504896    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/bc90f9c4-89a7-4b55-ada9-55c16dfad309-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:40:03.734078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.734039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:03.956478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.956387    6738 generic.go:356] "Generic (PLEG): container finished" podID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" containerID="a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b" exitCode=0
Oct 30 21:40:03.956986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.956729    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" event={"ID":"bc90f9c4-89a7-4b55-ada9-55c16dfad309","Type":"ContainerDied","Data":"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"}
Oct 30 21:40:03.956986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.956785    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4" event={"ID":"bc90f9c4-89a7-4b55-ada9-55c16dfad309","Type":"ContainerDied","Data":"c0fbf961d4e029d55ab26fd154b707ed4890994f5b220c2b56849b2e3be19b41"}
Oct 30 21:40:03.956986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.956815    6738 scope.go:122] "RemoveContainer" containerID="a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"
Oct 30 21:40:03.956986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:03.956987    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"
Oct 30 21:40:04.080044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.079978    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:40:04.126311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.126260    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-729802/payload-processing-external-model-e2e-extmod-a-729802-5fb75x7f4"]
Oct 30 21:40:04.163562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.163535    6738 scope.go:122] "RemoveContainer" containerID="a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"
Oct 30 21:40:04.165463 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:40:04.164184    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b\": container with ID starting with a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b not found: ID does not exist" containerID="a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"
Oct 30 21:40:04.165463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.164224    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b"} err="failed to get container status \"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b\": rpc error: code = NotFound desc = could not find container \"a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b\": container with ID starting with a44458165ccc071869a773c0db0743c5500c3edab58e869d91879de4f4741f1b not found: ID does not exist"
Oct 30 21:40:04.725975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.721973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:04.938418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:04.938389    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:05.091237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:05.091155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:05.165977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:05.165949    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" path="/var/lib/kubelet/pods/bc90f9c4-89a7-4b55-ada9-55c16dfad309/volumes"
Oct 30 21:40:05.645184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:05.645147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.452595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:06.452557    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.709681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:06.709609    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.842559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:06.842529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:06.966922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:06.966840    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:07.448675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:07.448637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:08.513955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:08.513923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:09.517468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:09.517272    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:10.442617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:10.442576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:11.327815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:11.327776    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:11.454115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:11.454084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:12.044404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:12.044369    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:12.337024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:12.336929    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:12.464040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:12.463998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.375233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:13.375200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.550188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:13.550152    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.686064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:13.685983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.829398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:13.829348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:13.983508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:13.983365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:14.289137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:14.289046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:14.459599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:14.459564    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:14.815100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:14.815047    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:15.013008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:15.012975    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:16.131216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:16.131183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:16.337743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:16.337714    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:17.077248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:17.077209    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:17.832481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:17.832421    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:18.564201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:18.564166    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:19.241537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:19.241502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:19.765121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:19.765091    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:19.898811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:19.898769    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:20.761361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:20.761326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:21.443566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:21.443525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:22.124647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:22.124616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:22.510733 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:40:22.510642    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:40:22.825026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:22.824942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:23.556643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:23.556613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:24.393155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:24.393122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:24.983973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:24.983940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:25.727475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:25.727424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:26.525037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:26.525000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:27.160891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:27.160862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:27.930039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:27.930007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:28.743367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:28.743335    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:29.969691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:29.969659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:30.854809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:30.854765    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:32.180094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:32.180000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:33.490766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:33.490728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:34.177114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:34.177082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:34.857444 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:34.857414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:35.532909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:35.532877    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:36.219531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:36.219491    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:36.951773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:36.951739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:37.624094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:37.624037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:38.341508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:38.341478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.125573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:39.125537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:39.854478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:39.854427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:40.641238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:40.641209    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:41.290299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:41.290266    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:42.050826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:42.050795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:42.759351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:42.759315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:43.525539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:43.525503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:44.214269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:44.214234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:45.057269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:45.057242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:45.721105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:45.721077    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:46.469052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:46.469015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:47.240838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:47.240788    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:47.893580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:47.893551    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:48.637916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:48.637859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:49.342582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:49.342542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:50.221129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:50.221080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:50.938780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:50.938750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:51.660200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:51.660170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:52.409420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:52.409390    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:52.529612 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:40:52.529550    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:40:53.043324 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:53.043287    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:53.723179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:53.723147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:54.433966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:54.433935    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:55.108386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:55.108354    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:55.783016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:55.782987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:56.450067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:56.450040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:57.133916 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:57.133884    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:57.948100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:57.948069    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:58.711029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:58.710996    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:40:59.352986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:40:59.352951    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:00.126497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:00.123264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:01.066168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:01.066140    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:01.837985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:01.837951    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:02.680570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:02.680542    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:03.496274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:03.496234    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:04.358102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:04.358074    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:05.063759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:05.063710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:05.835414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:05.835381    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:06.510590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:06.510562    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:07.189403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:07.189375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:07.769855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:07.769821    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:41:07.770073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:07.770049    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy" containerID="cri-o://fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46" gracePeriod=30
Oct 30 21:41:07.922527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:07.922494    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:08.691030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:08.690997    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:09.461156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:09.461117    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:10.255115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:10.255085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:11.033220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:11.033190    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:11.846014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:11.845983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:12.523316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:12.523285    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:13.082225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.082199    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:41:13.158829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.158793    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.159160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.158856    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.159160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.158891    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.159160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.158924    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.159973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.159756    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:13.160153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160110    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:13.160237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160170    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.160300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160265    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.160408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160335    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.160522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160502    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.160586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.160561    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\") pod \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\" (UID: \"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca\") "
Oct 30 21:41:13.161355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.161332    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.161518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.161357    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.161946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.161867    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:13.161946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.161921    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:13.162267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.162246    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:13.162984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.162963    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:13.164326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.163770    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:13.164326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.164073    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "kube-api-access-xzg8g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:13.164781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.164760    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token" pod "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" (UID: "6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:13.263020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.262985    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263019    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263034    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263047    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263064    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263077    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzg8g\" (UniqueName: \"kubernetes.io/projected/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-kube-api-access-xzg8g\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.263260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.263090    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:13.366217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.366182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:13.800488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.800389    6738 generic.go:356] "Generic (PLEG): container finished" podID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerID="fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46" exitCode=0
Oct 30 21:41:13.800488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.800478    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" event={"ID":"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca","Type":"ContainerDied","Data":"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"}
Oct 30 21:41:13.800846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.800484    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"
Oct 30 21:41:13.800846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.800523    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5" event={"ID":"6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca","Type":"ContainerDied","Data":"a6fab5f07f62607b7c0ee74666374bd6cb7473dd2ae4774e9d2e9729308de03b"}
Oct 30 21:41:13.800846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.800545    6738 scope.go:122] "RemoveContainer" containerID="fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"
Oct 30 21:41:13.816122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.816067    6738 scope.go:122] "RemoveContainer" containerID="fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"
Oct 30 21:41:13.816506 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:41:13.816414    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46\": container with ID starting with fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46 not found: ID does not exist" containerID="fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"
Oct 30 21:41:13.816637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.816501    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46"} err="failed to get container status \"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46\": rpc error: code = NotFound desc = could not find container \"fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46\": container with ID starting with fd87d4152ddf3b2c3683ff17483d3ff5bee2f010760f5dcd3ca0cd6444737f46 not found: ID does not exist"
Oct 30 21:41:13.833335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.833311    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:41:13.838402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:13.838378    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-pslk5"]
Oct 30 21:41:14.101655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:14.101560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:14.559933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:14.559901    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:14.932810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:14.932770    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:15.166386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:15.166349    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" path="/var/lib/kubelet/pods/6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca/volumes"
Oct 30 21:41:15.581719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:15.581675    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:16.283698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:16.283657    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:17.606462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:17.599023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:18.606715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:18.604148    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:20.035460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:20.031267    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:20.337555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:20.337465    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:21.351594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:21.351563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:22.129033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:22.128996    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:22.513449 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:41:22.513345    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:41:22.858399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:22.858317    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:23.630666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:23.630632    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:24.350592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:24.350561    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:25.169057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:25.169023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:25.342976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:25.342818    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:25.520537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:25.517326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.247250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:26.247177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.439314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:26.439274    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:26.557196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:26.557122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:27.087492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:27.087458    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:27.444507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:27.444421    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:28.149250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:28.149215    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:28.817567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:28.817529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:29.554664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:29.554635    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:30.287223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:30.287191    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:31.047622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:31.047583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:31.737577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:31.737540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:32.483866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:32.483826    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:32.765562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:32.764964    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:33.171810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:33.171771    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:33.289259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:33.289227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:33.538935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:33.538844    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:33.674576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:33.674537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:33.809233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:33.809144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:34.096509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:34.093196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:34.892535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:34.890110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:35.557417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:35.557384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:36.264845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:36.264807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:37.045643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:37.045602    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:37.836665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:37.836616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:38.570928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:38.570899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:39.355792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:39.354889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:40.218973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:40.218931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:40.735450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:40.735395    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:40.881463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:40.874235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:40.996273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:40.996193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:41.214911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:41.214876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:41.383170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:41.383136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:41.545457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:41.544979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:41.677771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:41.677695    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:42.015661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:42.015580    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:42.130769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:42.130732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:42.594595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:42.594563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:42.714459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:42.714264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:43.312990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:43.312918    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:43.442976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:43.442945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:44.265901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:44.265864    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:45.063399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:45.063367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:45.234838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:45.234789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:46.121003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:46.120971    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:46.870462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:46.870413    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:47.049448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:47.049400    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:47.196584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:47.196503    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:47.685374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:47.685337    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:47.994530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:47.994411    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:48.125357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:48.125318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:49.568196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:49.568096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:49.715483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:49.713606    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:49.923760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:49.923672    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:50.735773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:50.735735    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:41:50.736254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:50.736007    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy" containerID="cri-o://a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a" gracePeriod=30
Oct 30 21:41:50.740391 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:50.740364    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:41:50.740629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:50.740594    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy" containerID="cri-o://414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f" gracePeriod=30
Oct 30 21:41:51.336886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:51.336855    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:52.161661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:52.161617    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:52.469125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:52.469057    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:52.517478 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:41:52.517402    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:41:52.608318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:52.608281    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:52.775705 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:52.775625    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.085683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:53.085600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.204356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:53.204325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.706311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:53.706250    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:53.865377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:53.865328    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:54.457555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:54.457510    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:54.624137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:54.624090    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:54.857514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:54.853964    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:56.077723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.077692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:56.161947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.159882    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:41:56.221531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.221499    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:41:56.309117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309026    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309136    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.309284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309168    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.309284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309204    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309260    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309262    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.309521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309315    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309338    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309401    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309553    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309615    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309659    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\") pod \"96c0ef43-3948-49fd-973c-db6859e99e2d\" (UID: \"96c0ef43-3948-49fd-973c-db6859e99e2d\") "
Oct 30 21:41:56.309859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309687    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.309859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309698    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.309859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309725    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.309859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.309745    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.310147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310067    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.310229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310204    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.310288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310242    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.310348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310313    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.310664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310531    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:56.310780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.310761    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.311424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311400    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.311566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311428    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.311566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311485    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.311566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311500    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.311566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311513    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/96c0ef43-3948-49fd-973c-db6859e99e2d-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.311566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.311527    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.312114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.312086    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.312613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.312344    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.312726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.312704    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.313264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.313240    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:56.313358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.313337    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:56.313753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.313724    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "kube-api-access-n5r2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:56.313918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.313897    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:41:56.314607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.314577    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy" pod "96c0ef43-3948-49fd-973c-db6859e99e2d" (UID: "96c0ef43-3948-49fd-973c-db6859e99e2d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.314807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.314788    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:56.315333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.315316    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "kube-api-access-xjq55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:41:56.412520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.412405    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.412714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.412549    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\") pod \"3af20973-49cd-4eb8-9b78-035d466cfa18\" (UID: \"3af20973-49cd-4eb8-9b78-035d466cfa18\") "
Oct 30 21:41:56.412714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.412680    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:41:56.413129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413102    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert" pod "3af20973-49cd-4eb8-9b78-035d466cfa18" (UID: "3af20973-49cd-4eb8-9b78-035d466cfa18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:41:56.413380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413360    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413383    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3af20973-49cd-4eb8-9b78-035d466cfa18-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413397    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413410    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413422    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413482    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413497    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413511    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5r2g\" (UniqueName: \"kubernetes.io/projected/96c0ef43-3948-49fd-973c-db6859e99e2d-kube-api-access-n5r2g\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413524    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413538    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3af20973-49cd-4eb8-9b78-035d466cfa18-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413550    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.413563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.413563    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjq55\" (UniqueName: \"kubernetes.io/projected/3af20973-49cd-4eb8-9b78-035d466cfa18-kube-api-access-xjq55\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:41:56.765150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.765117    6738 generic.go:356] "Generic (PLEG): container finished" podID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerID="a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a" exitCode=0
Oct 30 21:41:56.765331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.765183    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"
Oct 30 21:41:56.765331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.765201    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" event={"ID":"96c0ef43-3948-49fd-973c-db6859e99e2d","Type":"ContainerDied","Data":"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"}
Oct 30 21:41:56.765331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.765263    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb" event={"ID":"96c0ef43-3948-49fd-973c-db6859e99e2d","Type":"ContainerDied","Data":"9f95ce67c5846d693f49886eb2403d27094e1a55de5c23e5ec2449b85d32b727"}
Oct 30 21:41:56.765331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.765293    6738 scope.go:122] "RemoveContainer" containerID="a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"
Oct 30 21:41:56.770060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.770038    6738 generic.go:356] "Generic (PLEG): container finished" podID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerID="414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f" exitCode=0
Oct 30 21:41:56.770356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.770323    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" event={"ID":"3af20973-49cd-4eb8-9b78-035d466cfa18","Type":"ContainerDied","Data":"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"}
Oct 30 21:41:56.770518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.770371    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj" event={"ID":"3af20973-49cd-4eb8-9b78-035d466cfa18","Type":"ContainerDied","Data":"e1567ee4f96624d9cc906aad7a27dc6b817bba028065b17432a09c156b0902f3"}
Oct 30 21:41:56.770518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.770467    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"
Oct 30 21:41:56.800104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.800089    6738 scope.go:122] "RemoveContainer" containerID="a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"
Oct 30 21:41:56.800417 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:41:56.800394    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a\": container with ID starting with a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a not found: ID does not exist" containerID="a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"
Oct 30 21:41:56.800507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.800447    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a"} err="failed to get container status \"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a\": rpc error: code = NotFound desc = could not find container \"a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a\": container with ID starting with a0ba7ad434290e9976b1aa3d73b8c77832849ff332db37cc681f75f538e68d1a not found: ID does not exist"
Oct 30 21:41:56.800507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.800473    6738 scope.go:122] "RemoveContainer" containerID="414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"
Oct 30 21:41:56.815598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.815424    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:41:56.815694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.815666    6738 scope.go:122] "RemoveContainer" containerID="414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"
Oct 30 21:41:56.816214 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:41:56.816171    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f\": container with ID starting with 414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f not found: ID does not exist" containerID="414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"
Oct 30 21:41:56.816329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.816208    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f"} err="failed to get container status \"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f\": rpc error: code = NotFound desc = could not find container \"414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f\": container with ID starting with 414a3f0c999b0585aed39d3035532a8b54ceb381248b159b0cbfb20d8a77812f not found: ID does not exist"
Oct 30 21:41:56.820154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.820132    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-48sbj"]
Oct 30 21:41:56.827693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.827667    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:41:56.835480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.835407    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-bwjrb"]
Oct 30 21:41:56.836496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:56.836474    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:57.168641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:57.168606    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" path="/var/lib/kubelet/pods/3af20973-49cd-4eb8-9b78-035d466cfa18/volumes"
Oct 30 21:41:57.169305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:57.169251    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" path="/var/lib/kubelet/pods/96c0ef43-3948-49fd-973c-db6859e99e2d/volumes"
Oct 30 21:41:57.628427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:57.628395    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:58.352829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:58.352781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:59.027673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:59.027632    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:41:59.730511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:41:59.730464    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:00.437646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:00.437614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:01.133578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:01.133537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:02.372007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:02.371972    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:02.540695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:02.540658    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:03.231933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:03.231893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:03.952707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:03.952671    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:04.658451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:04.658403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:05.323945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:05.323917    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:06.123936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:06.123895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:06.766002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:06.765968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:07.520257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:07.520224    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:08.244518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:08.244487    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:09.019128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:09.019101    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:09.837475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:09.837409    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:10.532853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:10.532803    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:11.228039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:11.227994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:11.954715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:11.954685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:12.718714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:12.718680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:13.504545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:13.504478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:14.347847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:14.347809    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:15.065828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:15.065800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:15.824410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:15.824379    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:16.478302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:16.478271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:17.175975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:17.175944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:18.130201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:18.130163    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:18.880417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:18.880382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:19.560717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:19.560661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.066392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:20.066361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.192680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:20.192608    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:20.920056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:20.920006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:21.840712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:21.840686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:22.515307 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:42:22.515262    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:42:22.536674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:22.536648    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:23.222786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:23.222751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:23.959400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:23.959356    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:24.352675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:24.352595    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:24.721641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:24.721611    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:25.415386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:25.415354    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:25.945724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:25.945696    6738 kubelet.go:1689] "Image garbage collection succeeded"
Oct 30 21:42:26.238784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:26.238703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:26.932026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:26.931992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:27.618731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:27.618686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:28.369606 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:28.369573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.138637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:29.138605    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:29.840062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:29.840021    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:30.612222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:30.612188    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:31.327416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:31.327376    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:31.992850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:31.992794    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:32.816975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:32.816939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:33.526265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:33.526236    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:34.170198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:34.170170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:35.049417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:35.049383    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:35.731641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:35.731612    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:36.505474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:36.505404    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:37.246563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:37.246500    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:38.045349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:38.045318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:38.821621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:38.821583    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:39.637528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:39.634837    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:40.252169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:40.252142    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:41.050647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:41.050616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:41.724534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:41.724501    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:42.395587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:42.395552    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:43.148124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:43.148092    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:43.992875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:43.992058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:44.775466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:44.770659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:45.583866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:45.583831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:46.235110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:46.235081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:46.956005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:46.955972    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:47.813727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:47.813692    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:47.941767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:47.941731    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:42:47.941945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:47.941925    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" containerID="cri-o://3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce" gracePeriod=30
Oct 30 21:42:48.629040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:48.628998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:49.541157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:49.541119    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:50.209798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:50.209761    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:50.971761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:50.971697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:51.704318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:51.704289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:51.936724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:51.936692    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:42:51.936970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:51.936946    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy" containerID="cri-o://edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586" gracePeriod=30
Oct 30 21:42:52.428812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:52.428778    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:52.548277 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:42:52.548238    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:42:53.152809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.152771    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:53.302553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.302517    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:42:53.366022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.365986    6738 generic.go:356] "Generic (PLEG): container finished" podID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerID="3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce" exitCode=0
Oct 30 21:42:53.366223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.366035    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"
Oct 30 21:42:53.366223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.366075    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" event={"ID":"5a870aa1-79b0-40e2-abf8-8e2c48176f6e","Type":"ContainerDied","Data":"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"}
Oct 30 21:42:53.366223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.366140    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" event={"ID":"5a870aa1-79b0-40e2-abf8-8e2c48176f6e","Type":"ContainerDied","Data":"0682d9f555da66bc7404f9577928328a6d5b854b567ea7e94e345767ad63aff2"}
Oct 30 21:42:53.366223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.366171    6738 scope.go:122] "RemoveContainer" containerID="3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"
Oct 30 21:42:53.378565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.378536    6738 scope.go:122] "RemoveContainer" containerID="3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"
Oct 30 21:42:53.378826 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:42:53.378809    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce\": container with ID starting with 3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce not found: ID does not exist" containerID="3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"
Oct 30 21:42:53.378894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.378835    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce"} err="failed to get container status \"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce\": rpc error: code = NotFound desc = could not find container \"3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce\": container with ID starting with 3f20e674412e125eee24d322e113b6b27baf58f8794543ee066f77da06f6d6ce not found: ID does not exist"
Oct 30 21:42:53.398656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.398634    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.398772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.398734    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.398832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.398768    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.398918    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.398957    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399048    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399058    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399098    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399129    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399192    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\") pod \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\" (UID: \"5a870aa1-79b0-40e2-abf8-8e2c48176f6e\") "
Oct 30 21:42:53.399420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399357    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:53.400024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.399626    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:53.400362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.400320    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:53.400565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.400544    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:53.400991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.400972    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.401086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.400996    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.401086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.401011    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.401086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.401024    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.401086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.401037    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.402796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.402774    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:53.403011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.402995    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:53.403641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.403616    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:53.403821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.403806    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk" pod "5a870aa1-79b0-40e2-abf8-8e2c48176f6e" (UID: "5a870aa1-79b0-40e2-abf8-8e2c48176f6e"). InnerVolumeSpecName "kube-api-access-7bwwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:53.502593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.502561    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bwwk\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-kube-api-access-7bwwk\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.502963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.502601    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.502963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.502614    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.502963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.502626    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a870aa1-79b0-40e2-abf8-8e2c48176f6e-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:53.701050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.700995    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:42:53.704419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.704391    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c"]
Oct 30 21:42:53.924527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:53.924498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:54.239301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:54.239257    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:42:54.239545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:54.239325    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-h7w8c" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:42:54.638759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:54.638732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:55.168167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:55.168126    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" path="/var/lib/kubelet/pods/5a870aa1-79b0-40e2-abf8-8e2c48176f6e/volumes"
Oct 30 21:42:55.329186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:55.329153    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:56.062243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:56.062167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:56.854331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:56.854303    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:57.230762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.230731    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:42:57.380768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380682    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.380941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380771    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.380941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380828    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.380941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380893    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380942    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.380974    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381177    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381305    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381389    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381395    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381462    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\") pod \"1e02ac66-a892-4357-aba3-c98cdef15dbd\" (UID: \"1e02ac66-a892-4357-aba3-c98cdef15dbd\") "
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381567    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381638    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:57.381799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.381763    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:57.382416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.382388    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e02ac66-a892-4357-aba3-c98cdef15dbd-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.382598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.382422    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.382598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.382485    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.382598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.382500    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.382598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.382513    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.383971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.383951    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:57.384629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.384608    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "kube-api-access-w7w74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:42:57.384777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.384751    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:42:57.385899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.385878    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy" pod "1e02ac66-a892-4357-aba3-c98cdef15dbd" (UID: "1e02ac66-a892-4357-aba3-c98cdef15dbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:42:57.424623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.424583    6738 generic.go:356] "Generic (PLEG): container finished" podID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerID="edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586" exitCode=0
Oct 30 21:42:57.424768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.424684    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" event={"ID":"1e02ac66-a892-4357-aba3-c98cdef15dbd","Type":"ContainerDied","Data":"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"}
Oct 30 21:42:57.424768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.424743    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz" event={"ID":"1e02ac66-a892-4357-aba3-c98cdef15dbd","Type":"ContainerDied","Data":"4bed2b504c1cd00ee8372f92c1cdd98f8984988e82d0a7cfa5d11c96734a26f1"}
Oct 30 21:42:57.424889 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.424775    6738 scope.go:122] "RemoveContainer" containerID="edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"
Oct 30 21:42:57.424944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.424934    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"
Oct 30 21:42:57.445354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.442578    6738 scope.go:122] "RemoveContainer" containerID="edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"
Oct 30 21:42:57.445685 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:42:57.445656    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586\": container with ID starting with edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586 not found: ID does not exist" containerID="edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"
Oct 30 21:42:57.445770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.445698    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586"} err="failed to get container status \"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586\": rpc error: code = NotFound desc = could not find container \"edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586\": container with ID starting with edf4b5f13201839b43d8a121593b3be549845ee8932b13620fb5c37171c32586 not found: ID does not exist"
Oct 30 21:42:57.451394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.451371    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:42:57.456863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.456842    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-5tkrz"]
Oct 30 21:42:57.484450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.484409    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.484544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.484452    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.484544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.484469    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w7w74\" (UniqueName: \"kubernetes.io/projected/1e02ac66-a892-4357-aba3-c98cdef15dbd-kube-api-access-w7w74\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.484544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.484483    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e02ac66-a892-4357-aba3-c98cdef15dbd-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:42:57.632972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:57.632894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:58.298483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:58.298426    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:59.002728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:59.002698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:42:59.169827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:59.169794    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" path="/var/lib/kubelet/pods/1e02ac66-a892-4357-aba3-c98cdef15dbd/volumes"
Oct 30 21:42:59.859294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:42:59.859265    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:00.547345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:00.547296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:01.357244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:01.357213    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:01.584040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:01.584007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:02.304862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:02.304817    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:02.970901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:02.970868    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:03.721044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:03.721014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:04.440658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:04.440586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:05.131871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:05.131835    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:05.922156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:05.922117    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:06.723880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:06.723851    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:07.714171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:07.714135    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:08.514674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:08.514637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:09.313696 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:09.313663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.021333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.021298    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.164886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.164854    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.295318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.295232    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.909076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.909042    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:10.936975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.936948    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:43:10.937734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937720    6738 cpu_manager.go:414] "containerMap: removing container" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy"
Oct 30 21:43:10.937791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937737    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy"
Oct 30 21:43:10.937791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937750    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy"
Oct 30 21:43:10.937791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937755    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy"
Oct 30 21:43:10.937791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937772    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy"
Oct 30 21:43:10.937791 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937777    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937815    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937820    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937829    6738 cpu_manager.go:414] "containerMap: removing container" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937834    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937844    6738 cpu_manager.go:414] "containerMap: removing container" podUID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" containerName="payload-processing"
Oct 30 21:43:10.937937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937848    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" containerName="payload-processing"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937947    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6418f2c2-9e0f-432f-bc5f-7c4f9a2b93ca" containerName="istio-proxy"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937963    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a870aa1-79b0-40e2-abf8-8e2c48176f6e" containerName="istio-proxy"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937970    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="96c0ef43-3948-49fd-973c-db6859e99e2d" containerName="istio-proxy"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937977    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc90f9c4-89a7-4b55-ada9-55c16dfad309" containerName="payload-processing"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937984    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e02ac66-a892-4357-aba3-c98cdef15dbd" containerName="istio-proxy"
Oct 30 21:43:10.938107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.937990    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3af20973-49cd-4eb8-9b78-035d466cfa18" containerName="istio-proxy"
Oct 30 21:43:10.943544 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.943525    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:10.950284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.950260    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"payload-processing-external-model-plugins-e2e-extmod-b-f1ed97\""
Oct 30 21:43:10.950401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.950288    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"routing-overlay\""
Oct 30 21:43:10.950401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.950288    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"kube-root-ca.crt\""
Oct 30 21:43:10.950401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.950344    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"payload-processing-external-model-e2e--650b641a-dockercfg-kd4ps\""
Oct 30 21:43:10.950564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.950549    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:43:10.951770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.951738    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"openshift-service-ca.crt\""
Oct 30 21:43:10.952475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:10.952460    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f1ed97\"/\"agc-cred-d071d120317f\""
Oct 30 21:43:11.029698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.029666    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.029888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.029711    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.029888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.029730    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8tbw\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.029888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.029882    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.135549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.134513    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.135549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.134591    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.135549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.134621    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8tbw\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.135549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.134877    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.137417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.137390    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.154526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.154497    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.162060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.160961    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.231621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.231188    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8tbw\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\") pod \"payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") " pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.265409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.265379    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:11.530370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.530313    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:11.658821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:11.658789    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:43:11.660321 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:43:11.660295    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3276351_bb91_4bbd_927d_294e8f9d8663.slice/crio-f7ebe4471df5b77d4f9f918e1ad447400851e8ae685dae9ddb80d9c26f525126 WatchSource:0}: Error finding container f7ebe4471df5b77d4f9f918e1ad447400851e8ae685dae9ddb80d9c26f525126: Status 404 returned error can't find the container with id f7ebe4471df5b77d4f9f918e1ad447400851e8ae685dae9ddb80d9c26f525126
Oct 30 21:43:12.427485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:12.427417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:12.623357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:12.623322    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" event={"ID":"e3276351-bb91-4bbd-927d-294e8f9d8663","Type":"ContainerStarted","Data":"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"}
Oct 30 21:43:12.623582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:12.623388    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:12.623582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:12.623407    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" event={"ID":"e3276351-bb91-4bbd-927d-294e8f9d8663","Type":"ContainerStarted","Data":"f7ebe4471df5b77d4f9f918e1ad447400851e8ae685dae9ddb80d9c26f525126"}
Oct 30 21:43:12.650308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:12.650261    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" podStartSLOduration=2.650247038 podStartE2EDuration="2.650247038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:43:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:43:12.644844846 +0000 UTC m=+2449.600532073" watchObservedRunningTime="2026-10-30 21:43:12.650247038 +0000 UTC m=+2449.605934270"
Oct 30 21:43:13.238684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:13.238637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:13.943146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:13.943112    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:14.770573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:14.770533    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:15.549961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:15.549928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:16.252929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:16.252902    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:16.950757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:16.950728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:17.766784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:17.766752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:18.529870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:18.529839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:19.324234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:19.324195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:20.099237 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:20.099208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:20.226809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:20.226779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:20.951614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:20.951586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:21.636194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:21.636159    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:22.390478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:22.388259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:22.513218 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:43:22.513174    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:43:23.274950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:23.274910    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:23.636350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:23.636320    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:43:24.033222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:24.033147    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:25.044821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:25.044789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:25.842244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:25.842210    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:25.978684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:25.978655    6738 scope.go:122] "RemoveContainer" containerID="cd2ca3db97d7fd61cee10e88048072cdcb7b35b3d74718b1b2bfa0a5b2fabc35"
Oct 30 21:43:26.626684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:26.626647    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:27.439424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:27.439389    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:28.203580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:28.203535    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:29.024761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:29.024721    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:29.810614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:29.810577    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:30.654337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:30.654305    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:31.550374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:31.550310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.267193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:32.267143    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:32.975675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:32.975643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:33.788465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:33.784970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:34.531972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:34.531921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:35.276634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:35.276598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:36.287535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:36.287496    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:37.249417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:37.249386    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:38.202897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:38.202862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:39.036463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:39.017348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:39.860955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:39.860908    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:40.774471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:40.770853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:41.712463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:41.711754    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:42.596333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:42.596299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:43.557561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:43.557518    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:44.529202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:44.529170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:45.392377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:45.392346    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:46.429348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:46.426922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:47.372461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:47.371466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:48.475479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:48.475018    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:49.388618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:49.388573    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:50.297459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:50.288103    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:51.243180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:51.243027    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:52.121338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:52.121307    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:52.340797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:52.340764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:52.512448 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:43:52.512351    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:43:53.232610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:53.232577    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:54.026485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:54.026427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:54.909658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:54.909607    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:55.895461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:55.887157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:56.719097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:56.719058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:57.599462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:57.597114    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:58.579821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:58.579787    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:43:59.474147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:43:59.474116    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:00.345158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:00.345123    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:01.153052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:01.153007    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:01.932607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:01.932571    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:02.774380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:02.774350    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:03.558366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:03.558318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:04.408477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:04.408393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:05.369460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:05.368035    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:06.169076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:06.169044    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:06.900033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:06.899992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:07.794217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:07.794183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:08.594981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:08.594950    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:09.386464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:09.384109    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:10.154862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:10.154832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:11.059142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:11.059096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:12.075427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:12.075391    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:13.098991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:13.098958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.059605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:14.059572    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:14.980735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:14.977784    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:16.035571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:16.035508    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:16.925971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:16.925939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:17.763197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:17.763168    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:18.525893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:18.525862    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:19.515687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:19.515640    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:20.206856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:20.206818    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:20.345036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:20.344994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:21.061014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:21.060983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:21.921268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:21.921117    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:22.515886 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:44:22.515845    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:44:22.818398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:22.815010    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:23.468073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:23.468037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:24.255614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:24.255568    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.061750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:25.061718    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.436158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:25.436109    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:44:25.465672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:25.465646    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:25.633128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:25.633094    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:26.395462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:26.388025    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:27.246283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:27.246251    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:28.030420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:28.030333    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:29.005950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:29.005912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:30.106020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.105984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:30.437032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.436990    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" podUID="e3276351-bb91-4bbd-927d-294e8f9d8663" containerName="payload-processing" containerID="cri-o://b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d" gracePeriod=25
Oct 30 21:44:30.743939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.743913    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:44:30.846923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.846895    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\") pod \"e3276351-bb91-4bbd-927d-294e8f9d8663\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") "
Oct 30 21:44:30.847116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.846938    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\") pod \"e3276351-bb91-4bbd-927d-294e8f9d8663\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") "
Oct 30 21:44:30.847116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.847042    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\") pod \"e3276351-bb91-4bbd-927d-294e8f9d8663\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") "
Oct 30 21:44:30.847116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.847084    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\") pod \"e3276351-bb91-4bbd-927d-294e8f9d8663\" (UID: \"e3276351-bb91-4bbd-927d-294e8f9d8663\") "
Oct 30 21:44:30.847352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.847321    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume" pod "e3276351-bb91-4bbd-927d-294e8f9d8663" (UID: "e3276351-bb91-4bbd-927d-294e8f9d8663"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:30.847825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.847748    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume" pod "e3276351-bb91-4bbd-927d-294e8f9d8663" (UID: "e3276351-bb91-4bbd-927d-294e8f9d8663"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:30.848178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.848154    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:30.848263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.848186    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3276351-bb91-4bbd-927d-294e8f9d8663-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:30.849355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.849325    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw" pod "e3276351-bb91-4bbd-927d-294e8f9d8663" (UID: "e3276351-bb91-4bbd-927d-294e8f9d8663"). InnerVolumeSpecName "kube-api-access-d8tbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:30.849488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.849417    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials" pod "e3276351-bb91-4bbd-927d-294e8f9d8663" (UID: "e3276351-bb91-4bbd-927d-294e8f9d8663"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:30.951165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.950067    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8tbw\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-kube-api-access-d8tbw\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:30.951165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.950111    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/e3276351-bb91-4bbd-927d-294e8f9d8663-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:30.984734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:30.984645    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.373760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.373718    6738 generic.go:356] "Generic (PLEG): container finished" podID="e3276351-bb91-4bbd-927d-294e8f9d8663" containerID="b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d" exitCode=0
Oct 30 21:44:31.374262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.373828    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" event={"ID":"e3276351-bb91-4bbd-927d-294e8f9d8663","Type":"ContainerDied","Data":"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"}
Oct 30 21:44:31.374262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.373888    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw" event={"ID":"e3276351-bb91-4bbd-927d-294e8f9d8663","Type":"ContainerDied","Data":"f7ebe4471df5b77d4f9f918e1ad447400851e8ae685dae9ddb80d9c26f525126"}
Oct 30 21:44:31.374262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.373833    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"
Oct 30 21:44:31.374262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.373924    6738 scope.go:122] "RemoveContainer" containerID="b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"
Oct 30 21:44:31.393099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.393081    6738 scope.go:122] "RemoveContainer" containerID="b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"
Oct 30 21:44:31.393387 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:44:31.393358    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d\": container with ID starting with b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d not found: ID does not exist" containerID="b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"
Oct 30 21:44:31.393589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.393388    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d"} err="failed to get container status \"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d\": rpc error: code = NotFound desc = could not find container \"b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d\": container with ID starting with b5c0bfbf84259e38da573861e8ff05fff556bb49ec9f2525afeae646289a893d not found: ID does not exist"
Oct 30 21:44:31.403958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.403931    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:31.430478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.429419    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:44:31.439866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.439838    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-f1ed97/payload-processing-external-model-e2e-extmod-b-f1ed97-79f9fpxkw"]
Oct 30 21:44:31.584244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:31.584159    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:32.934493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:32.931988    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:33.166297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.166267    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3276351-bb91-4bbd-927d-294e8f9d8663" path="/var/lib/kubelet/pods/e3276351-bb91-4bbd-927d-294e8f9d8663/volumes"
Oct 30 21:44:33.295571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.295497    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:44:33.296320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.296306    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e3276351-bb91-4bbd-927d-294e8f9d8663" containerName="payload-processing"
Oct 30 21:44:33.296377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.296321    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3276351-bb91-4bbd-927d-294e8f9d8663" containerName="payload-processing"
Oct 30 21:44:33.296521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.296503    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3276351-bb91-4bbd-927d-294e8f9d8663" containerName="payload-processing"
Oct 30 21:44:33.301959 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.301939    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.309252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.309226    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:44:33.310321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310286    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"payload-processing-external-model-plugins-e2e-extmod-b-36a099\""
Oct 30 21:44:33.310498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310360    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"routing-overlay\""
Oct 30 21:44:33.310498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310396    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"agc-cred-047e8af88945\""
Oct 30 21:44:33.310498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310421    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"agc-cred-733bb027a41d\""
Oct 30 21:44:33.310498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310472    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"kube-root-ca.crt\""
Oct 30 21:44:33.310692 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.310506    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"openshift-service-ca.crt\""
Oct 30 21:44:33.313633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.313613    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-36a099\"/\"payload-processing-external-model-e2e--3e617b48-dockercfg-5bf72\""
Oct 30 21:44:33.381694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.381668    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.381841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.381697    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.381902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.381877    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2wz\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.382138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.382119    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.483210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.483182    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.483361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.483287    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.483361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.483318    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.483528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.483405    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2wz\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.484113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.484083    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.484206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.484085    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.486190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.486165    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.493913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.493888    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2wz\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.621958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.621925    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:33.927106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:33.927076    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:44:33.929227 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:44:33.929199    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ee710b_c77f_4452_a567_bd5ff944fac4.slice/crio-4a26b85cdecd48d0cc8cf3320f55d9d8ba8fac4dc923ca6788cd5ccbc35db52e WatchSource:0}: Error finding container 4a26b85cdecd48d0cc8cf3320f55d9d8ba8fac4dc923ca6788cd5ccbc35db52e: Status 404 returned error can't find the container with id 4a26b85cdecd48d0cc8cf3320f55d9d8ba8fac4dc923ca6788cd5ccbc35db52e
Oct 30 21:44:34.070729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.070697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:34.482740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.482653    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" event={"ID":"74ee710b-c77f-4452-a567-bd5ff944fac4","Type":"ContainerStarted","Data":"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"}
Oct 30 21:44:34.482740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.482720    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:34.482740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.482737    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" event={"ID":"74ee710b-c77f-4452-a567-bd5ff944fac4","Type":"ContainerStarted","Data":"4a26b85cdecd48d0cc8cf3320f55d9d8ba8fac4dc923ca6788cd5ccbc35db52e"}
Oct 30 21:44:34.513691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.513634    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" podStartSLOduration=1.513615935 podStartE2EDuration="1.513615935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:44:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:44:34.513349828 +0000 UTC m=+2531.469037044" watchObservedRunningTime="2026-10-30 21:44:34.513615935 +0000 UTC m=+2531.469303164"
Oct 30 21:44:34.985460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:34.982863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:35.789726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:35.789667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:36.092495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:36.092404    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:44:36.092680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:36.092659    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy" containerID="cri-o://f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1" gracePeriod=30
Oct 30 21:44:36.800953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:36.797974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:37.586098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:37.586056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:37.760466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:37.759360    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:38.219228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:38.219194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:38.382807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:38.382773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:39.073128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:39.071585    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:39.411239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:39.411208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:39.894941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:39.894868    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:40.667421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:40.667379    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:41.391576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.391549    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:44:41.502805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.502771    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535292    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535494    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535613    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535640    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535749    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535779    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535872    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.535979    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.536046    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\") pod \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\" (UID: \"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e\") "
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.543417    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.544646    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.544714    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.547889    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.547918    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.552466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.547933    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.568808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.561157    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:44:41.568808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.567622    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:41.592801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.588693    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:41.620802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.620266    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:44:41.624101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.624067    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "kube-api-access-pvr99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:44:41.624991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.624599    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo" pod "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" (UID: "12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:44:41.634309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.631252    6738 generic.go:356] "Generic (PLEG): container finished" podID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerID="f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1" exitCode=0
Oct 30 21:44:41.634309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.631413    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" event={"ID":"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e","Type":"ContainerDied","Data":"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"}
Oct 30 21:44:41.634309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.631471    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg" event={"ID":"12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e","Type":"ContainerDied","Data":"4f7044dad978909cf5bfdb9b0e7a5a5f1f750f8b38df51d06c4e53d26a2f2ff5"}
Oct 30 21:44:41.634309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.631501    6738 scope.go:122] "RemoveContainer" containerID="f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"
Oct 30 21:44:41.634309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.631666    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"
Oct 30 21:44:41.648688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.648671    6738 scope.go:122] "RemoveContainer" containerID="f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"
Oct 30 21:44:41.649126 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:44:41.649100    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1\": container with ID starting with f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1 not found: ID does not exist" containerID="f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"
Oct 30 21:44:41.649214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649133    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1"} err="failed to get container status \"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1\": rpc error: code = NotFound desc = could not find container \"f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1\": container with ID starting with f7d3bd59fce56659ce21692a303f6f5226b250339fc70c4588b28112e21cdcc1 not found: ID does not exist"
Oct 30 21:44:41.649362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649342    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.649779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649369    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvr99\" (UniqueName: \"kubernetes.io/projected/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-kube-api-access-pvr99\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.649779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649383    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.649779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649396    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.649779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649408    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.649779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.649421    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:44:41.660748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.660723    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:44:41.666509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:41.666486    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-szflg"]
Oct 30 21:44:42.388119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:42.388034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:43.167707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:43.167667    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" path="/var/lib/kubelet/pods/12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e/volumes"
Oct 30 21:44:43.288043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:43.288010    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:44.177744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:44.177690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:45.034460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:45.025759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:45.496957 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:45.496931    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:44:45.947088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:45.947043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:46.147472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:46.147405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:46.577126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:46.577045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:46.723845 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:46.723796    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:46.919021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:46.918983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:47.251783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:47.251667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:47.529501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:47.529383    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:47.740340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:47.740302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:48.994955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:48.994922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:49.220517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:49.220416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:49.737329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:49.737302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:50.875322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:50.875282    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:51.087932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:51.087899    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:51.301518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:51.297992    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:51.590318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:51.584845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:52.594556 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:44:52.593630    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:44:53.077338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:53.077300    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.491948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:53.491904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:53.824539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:53.824421    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:55.113463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:55.107936    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:55.387029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:55.386820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:55.612274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:55.612244    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:56.891040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:56.890984    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:57.328325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:57.328203    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:58.100938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:58.100908    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.270419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:59.270385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:44:59.429966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:44:59.429928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:00.061402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.061373    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:00.380754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.380721    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:00.381610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.381592    6738 cpu_manager.go:414] "containerMap: removing container" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy"
Oct 30 21:45:00.381658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.381612    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy"
Oct 30 21:45:00.381769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.381758    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="12dc4ca0-ee6f-48c5-8959-88b05e2f8d5e" containerName="istio-proxy"
Oct 30 21:45:00.385716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.385700    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.409714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.409684    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:00.412406 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.412366    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lrk2v\""
Oct 30 21:45:00.443213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.442419    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:00.448177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.448153    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.456579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.456540    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.456722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.456654    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbbn6\" (UniqueName: \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.456796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.456759    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbgwq\" (UniqueName: \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.457771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.457744    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.485173 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.485144    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:00.498521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.498485    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:00.504403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.504382    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.537796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.537771    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:00.543967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.543942    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:00.547563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.547540    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.549038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.549016    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:00.549204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.549111    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.558851    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.558891    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.559705    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.559790    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.559845    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdz8k\" (UniqueName: \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.559912    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbbn6\" (UniqueName: \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.559994    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.560180    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbgwq\" (UniqueName: \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.560271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.560242    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfrq5\" (UniqueName: \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.561008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.560304    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmn6d\" (UniqueName: \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.561008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.560357    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.565112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.561916    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.576199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.576169    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:00.584548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.584484    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:00.591285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.591255    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbbn6\" (UniqueName: \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\") pod \"maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.599428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.599399    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbgwq\" (UniqueName: \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\") pod \"maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.662335 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662258    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdz8k\" (UniqueName: \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.662492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662385    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.662550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662507    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfrq5\" (UniqueName: \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.662622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662574    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmn6d\" (UniqueName: \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.662682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662623    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.662735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.662692    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.663211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.663180    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.663517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.663490    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.663651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.663525    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.695849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.695734    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfrq5\" (UniqueName: \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\") pod \"maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:00.695963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.695904    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdz8k\" (UniqueName: \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\") pod \"maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.707752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.707702    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmn6d\" (UniqueName: \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\") pod \"maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.712516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.712492    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:00.766014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.765828    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:00.827059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.826730    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:00.888614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.888365    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:00.933221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:00.933192    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:01.014170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.014141    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:01.087963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.087635    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerStarted","Data":"e42be9777db507c6b555f5e6913edd5804e622d314d30d2f0a2d9851ee59d45c"}
Oct 30 21:45:01.103165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.103130    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:01.221521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.221001    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:01.299978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.299896    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:01.303363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.303333    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:01.579614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:01.579586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:02.108817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.108731    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerStarted","Data":"68f591c380db7ef30074150597a30d93ee2c29d637b2da478073c2abcd3169a7"}
Oct 30 21:45:02.109258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.109147    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerStarted","Data":"ec1e3f88357a0c5de9c289e5d422af91f9f08813c71a78fd1c03506f451813df"}
Oct 30 21:45:02.111601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.111567    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerStarted","Data":"04b9821f9d17c2cab748c3911676cd7ffe527ffc2488754a86980df0b98886dd"}
Oct 30 21:45:02.114040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.113792    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerStarted","Data":"2692e67e13009b417bd7595873d4b12c8ef63100796f3254291309de82683056"}
Oct 30 21:45:02.114040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.113854    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerStarted","Data":"c5f4f39346a7accb4b6b126e8e119dddf132aa6d8cc9e1e584754e0b029d76dc"}
Oct 30 21:45:02.115919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.115888    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerStarted","Data":"f67a2b514781ab9deef6d8eced4592484a639dcdd5580a17384d9509875bbf3b"}
Oct 30 21:45:02.116013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.115941    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerStarted","Data":"b4f0057668cd9cab3dc6bd66a66522bb2513696ff4eae9a7f772e2d3c5efaa64"}
Oct 30 21:45:02.118309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.118277    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerStarted","Data":"309a8f74073d383427bbd2b8027ff1fb10a6971b7977af4afb6f24e1cc31446b"}
Oct 30 21:45:02.118403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.118325    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerStarted","Data":"1e7e4264de1c90cf3a2914d5c1523798402a26e5d410b4e89f3ed75f07cdd4ee"}
Oct 30 21:45:02.137370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.137329    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" podStartSLOduration=2.137317757 podStartE2EDuration="2.137317757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.131594179 +0000 UTC m=+2559.087281406" watchObservedRunningTime="2026-10-30 21:45:02.137317757 +0000 UTC m=+2559.093004983"
Oct 30 21:45:02.158322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.158283    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" podStartSLOduration=2.15827114 podStartE2EDuration="2.15827114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.156926507 +0000 UTC m=+2559.112613733" watchObservedRunningTime="2026-10-30 21:45:02.15827114 +0000 UTC m=+2559.113958394"
Oct 30 21:45:02.174422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.174383    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" podStartSLOduration=2.1743725720000002 podStartE2EDuration="2.174372572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.172984979 +0000 UTC m=+2559.128672206" watchObservedRunningTime="2026-10-30 21:45:02.174372572 +0000 UTC m=+2559.130059793"
Oct 30 21:45:02.218111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.218050    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" podStartSLOduration=2.218034045 podStartE2EDuration="2.218034045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.197898357 +0000 UTC m=+2559.153585583" watchObservedRunningTime="2026-10-30 21:45:02.218034045 +0000 UTC m=+2559.173721265"
Oct 30 21:45:02.220331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.220295    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" podStartSLOduration=2.220285288 podStartE2EDuration="2.220285288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:02.217010316 +0000 UTC m=+2559.172697543" watchObservedRunningTime="2026-10-30 21:45:02.220285288 +0000 UTC m=+2559.175972515"
Oct 30 21:45:02.522260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.522178    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:02.933006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:02.932971    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:03.073258 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.073223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:03.151246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.151210    6738 generic.go:356] "Generic (PLEG): container finished" podID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerID="04b9821f9d17c2cab748c3911676cd7ffe527ffc2488754a86980df0b98886dd" exitCode=60
Oct 30 21:45:03.151614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.151584    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerDied","Data":"04b9821f9d17c2cab748c3911676cd7ffe527ffc2488754a86980df0b98886dd"}
Oct 30 21:45:03.152341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.152319    6738 scope.go:122] "RemoveContainer" containerID="04b9821f9d17c2cab748c3911676cd7ffe527ffc2488754a86980df0b98886dd"
Oct 30 21:45:03.208898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.208805    6738 generic.go:356] "Generic (PLEG): container finished" podID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerID="2692e67e13009b417bd7595873d4b12c8ef63100796f3254291309de82683056" exitCode=60
Oct 30 21:45:03.212581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.212561    6738 generic.go:356] "Generic (PLEG): container finished" podID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerID="f67a2b514781ab9deef6d8eced4592484a639dcdd5580a17384d9509875bbf3b" exitCode=60
Oct 30 21:45:03.214779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.214760    6738 scope.go:122] "RemoveContainer" containerID="f67a2b514781ab9deef6d8eced4592484a639dcdd5580a17384d9509875bbf3b"
Oct 30 21:45:03.215756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.215738    6738 scope.go:122] "RemoveContainer" containerID="2692e67e13009b417bd7595873d4b12c8ef63100796f3254291309de82683056"
Oct 30 21:45:03.216855 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.216819    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerDied","Data":"2692e67e13009b417bd7595873d4b12c8ef63100796f3254291309de82683056"}
Oct 30 21:45:03.217909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.217888    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerDied","Data":"f67a2b514781ab9deef6d8eced4592484a639dcdd5580a17384d9509875bbf3b"}
Oct 30 21:45:03.219142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.219127    6738 generic.go:356] "Generic (PLEG): container finished" podID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerID="309a8f74073d383427bbd2b8027ff1fb10a6971b7977af4afb6f24e1cc31446b" exitCode=60
Oct 30 21:45:03.219392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.219374    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerDied","Data":"309a8f74073d383427bbd2b8027ff1fb10a6971b7977af4afb6f24e1cc31446b"}
Oct 30 21:45:03.219914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.219868    6738 scope.go:122] "RemoveContainer" containerID="309a8f74073d383427bbd2b8027ff1fb10a6971b7977af4afb6f24e1cc31446b"
Oct 30 21:45:03.222512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.222488    6738 generic.go:356] "Generic (PLEG): container finished" podID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerID="68f591c380db7ef30074150597a30d93ee2c29d637b2da478073c2abcd3169a7" exitCode=60
Oct 30 21:45:03.222617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.222601    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerDied","Data":"68f591c380db7ef30074150597a30d93ee2c29d637b2da478073c2abcd3169a7"}
Oct 30 21:45:03.223017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:03.222991    6738 scope.go:122] "RemoveContainer" containerID="68f591c380db7ef30074150597a30d93ee2c29d637b2da478073c2abcd3169a7"
Oct 30 21:45:04.234674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.233401    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerStarted","Data":"faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"}
Oct 30 21:45:04.236026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.235993    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerStarted","Data":"f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"}
Oct 30 21:45:04.238294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.238270    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerStarted","Data":"dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"}
Oct 30 21:45:04.240520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.240494    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerStarted","Data":"4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"}
Oct 30 21:45:04.242634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.242610    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerStarted","Data":"c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"}
Oct 30 21:45:04.356489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:04.356448    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.072842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.072786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.206129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.206056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.253052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.253022    6738 generic.go:356] "Generic (PLEG): container finished" podID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerID="faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b" exitCode=60
Oct 30 21:45:05.253587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.253134    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerDied","Data":"faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"}
Oct 30 21:45:05.253587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.253383    6738 scope.go:122] "RemoveContainer" containerID="2692e67e13009b417bd7595873d4b12c8ef63100796f3254291309de82683056"
Oct 30 21:45:05.253587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.253467    6738 scope.go:122] "RemoveContainer" containerID="faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"
Oct 30 21:45:05.253918 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:05.253879    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt_odh-ai-gateway-infra(3cbbcfd4-34ea-4c0d-8489-764092364dc4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4"
Oct 30 21:45:05.256084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.255455    6738 generic.go:356] "Generic (PLEG): container finished" podID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerID="f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0" exitCode=60
Oct 30 21:45:05.256084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.255563    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerDied","Data":"f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"}
Oct 30 21:45:05.256084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.255847    6738 scope.go:122] "RemoveContainer" containerID="f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"
Oct 30 21:45:05.256270 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:05.256178    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f_odh-ai-gateway-infra(e6f803ab-d562-41e6-a257-ca8b1d1a310f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f"
Oct 30 21:45:05.257833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.257810    6738 generic.go:356] "Generic (PLEG): container finished" podID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerID="dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56" exitCode=60
Oct 30 21:45:05.257971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.257956    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerDied","Data":"dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"}
Oct 30 21:45:05.258288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.258270    6738 scope.go:122] "RemoveContainer" containerID="dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"
Oct 30 21:45:05.258598 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:05.258580    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8_odh-ai-gateway-infra(002dd75c-389b-4eb0-b1b7-9bc2ba42804c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c"
Oct 30 21:45:05.260521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.260499    6738 generic.go:356] "Generic (PLEG): container finished" podID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerID="4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051" exitCode=60
Oct 30 21:45:05.260621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.260573    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerDied","Data":"4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"}
Oct 30 21:45:05.260760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.260741    6738 scope.go:122] "RemoveContainer" containerID="4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"
Oct 30 21:45:05.261008 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:05.260986    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs_odh-ai-gateway-infra(daf6bc60-abf2-4196-926d-4bbbceeb099e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e"
Oct 30 21:45:05.262864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.262841    6738 generic.go:356] "Generic (PLEG): container finished" podID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerID="c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500" exitCode=60
Oct 30 21:45:05.262955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.262886    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerDied","Data":"c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"}
Oct 30 21:45:05.263123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.263111    6738 scope.go:122] "RemoveContainer" containerID="c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"
Oct 30 21:45:05.263350 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:05.263333    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd_odh-ai-gateway-infra(2e50bf6e-2a48-4f4c-abf5-0cf005499a81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81"
Oct 30 21:45:05.271648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.271630    6738 scope.go:122] "RemoveContainer" containerID="f67a2b514781ab9deef6d8eced4592484a639dcdd5580a17384d9509875bbf3b"
Oct 30 21:45:05.492456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.470950    6738 scope.go:122] "RemoveContainer" containerID="309a8f74073d383427bbd2b8027ff1fb10a6971b7977af4afb6f24e1cc31446b"
Oct 30 21:45:05.652272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.652248    6738 scope.go:122] "RemoveContainer" containerID="68f591c380db7ef30074150597a30d93ee2c29d637b2da478073c2abcd3169a7"
Oct 30 21:45:05.675316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.675123    6738 scope.go:122] "RemoveContainer" containerID="04b9821f9d17c2cab748c3911676cd7ffe527ffc2488754a86980df0b98886dd"
Oct 30 21:45:05.746053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.745955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:05.937268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:05.937164    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:06.332383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.330511    6738 scope.go:122] "RemoveContainer" containerID="4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"
Oct 30 21:45:06.332383 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:06.330941    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs_odh-ai-gateway-infra(daf6bc60-abf2-4196-926d-4bbbceeb099e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e"
Oct 30 21:45:06.381459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.380787    6738 scope.go:122] "RemoveContainer" containerID="c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"
Oct 30 21:45:06.389479 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:06.383733    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd_odh-ai-gateway-infra(2e50bf6e-2a48-4f4c-abf5-0cf005499a81)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81"
Oct 30 21:45:06.397458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.393635    6738 scope.go:122] "RemoveContainer" containerID="faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"
Oct 30 21:45:06.397458 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:06.394074    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt_odh-ai-gateway-infra(3cbbcfd4-34ea-4c0d-8489-764092364dc4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4"
Oct 30 21:45:06.418456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.414867    6738 scope.go:122] "RemoveContainer" containerID="f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"
Oct 30 21:45:06.418456 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:06.415326    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f_odh-ai-gateway-infra(e6f803ab-d562-41e6-a257-ca8b1d1a310f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f"
Oct 30 21:45:06.423627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.421673    6738 scope.go:122] "RemoveContainer" containerID="dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"
Oct 30 21:45:06.423627 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:06.422087    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8_odh-ai-gateway-infra(002dd75c-389b-4eb0-b1b7-9bc2ba42804c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c"
Oct 30 21:45:06.953998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:06.953965    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:45:07.083814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.076267    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.083814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.078612    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:45:07.089010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.088850    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"payload-processing-external-model-plugins-e2e-extmod-a-fb2b58\""
Oct 30 21:45:07.089010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.088974    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"agc-cred-98afef9e5c37\""
Oct 30 21:45:07.089132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.089100    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"payload-processing-external-model-e2e--b5c8139c-dockercfg-b6q5w\""
Oct 30 21:45:07.089189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.089171    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"agc-cred-b3a133238e6c\""
Oct 30 21:45:07.089285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.089270    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"kube-root-ca.crt\""
Oct 30 21:45:07.089378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.089363    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"routing-overlay\""
Oct 30 21:45:07.089520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.089449    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-fb2b58\"/\"openshift-service-ca.crt\""
Oct 30 21:45:07.206836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.206747    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.206967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.206853    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.207044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.207030    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wlzj\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.207109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.207096    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.308040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308006    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wlzj\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.308187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308073    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.308187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308155    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.308283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308193    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.308794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308767    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.309033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.308987    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.311012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.310986    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.325071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.325051    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wlzj\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\") pod \"payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") " pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.403755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.403704    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:07.480888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.478917    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:45:07.499479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.496997    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.499479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.496894    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.501972    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"routing-overlay\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502025    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-plugins-e2e-extmod-b-0ea1ea\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502084    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-5662daf86227\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502157    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"openshift-service-ca.crt\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502212    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"kube-root-ca.crt\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502275    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"agc-cred-00b228f0efa6\""
Oct 30 21:45:07.503484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.502354    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-0ea1ea\"/\"payload-processing-external-model-e2e--8e1d66cd-dockercfg-d8wks\""
Oct 30 21:45:07.521608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.521543    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.521608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.521598    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.521875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.521676    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.521875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.521726    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bthd\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.623556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.623505    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.623775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.623573    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.623775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.623650    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.623775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.623704    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bthd\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.624857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.624810    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.625002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.624946    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.626500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.626475    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.633452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.633419    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bthd\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\") pod \"payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") " pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.700183 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:45:07.700153    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5627795_0bb4_4ece_89a6_75bb7bd67da7.slice/crio-730a64eff68a5dbb2357857db631ad4e180af33e7b9a9b8fa0012d3c8305304b WatchSource:0}: Error finding container 730a64eff68a5dbb2357857db631ad4e180af33e7b9a9b8fa0012d3c8305304b: Status 404 returned error can't find the container with id 730a64eff68a5dbb2357857db631ad4e180af33e7b9a9b8fa0012d3c8305304b
Oct 30 21:45:07.700283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.700188    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:45:07.833005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.832975    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:07.934041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:07.933138    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:08.199764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.199733    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:45:08.200871 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:45:08.200841    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a633f34_b01b_464a_8747_22120dab9b9b.slice/crio-0a792bdf47662cba0c995ed0acab10398d3930aec82209c73f4919e818de08e8 WatchSource:0}: Error finding container 0a792bdf47662cba0c995ed0acab10398d3930aec82209c73f4919e818de08e8: Status 404 returned error can't find the container with id 0a792bdf47662cba0c995ed0acab10398d3930aec82209c73f4919e818de08e8
Oct 30 21:45:08.438869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.438820    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" event={"ID":"9a633f34-b01b-464a-8747-22120dab9b9b","Type":"ContainerStarted","Data":"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"}
Oct 30 21:45:08.439311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.438880    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" event={"ID":"9a633f34-b01b-464a-8747-22120dab9b9b","Type":"ContainerStarted","Data":"0a792bdf47662cba0c995ed0acab10398d3930aec82209c73f4919e818de08e8"}
Oct 30 21:45:08.439311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.439060    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:08.440902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.440879    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" event={"ID":"f5627795-0bb4-4ece-89a6-75bb7bd67da7","Type":"ContainerStarted","Data":"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"}
Oct 30 21:45:08.441052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.440919    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" event={"ID":"f5627795-0bb4-4ece-89a6-75bb7bd67da7","Type":"ContainerStarted","Data":"730a64eff68a5dbb2357857db631ad4e180af33e7b9a9b8fa0012d3c8305304b"}
Oct 30 21:45:08.441052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.440971    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:08.459869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.459826    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" podStartSLOduration=1.45981066 podStartE2EDuration="1.45981066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:08.458681254 +0000 UTC m=+2565.414368481" watchObservedRunningTime="2026-10-30 21:45:08.45981066 +0000 UTC m=+2565.415497937"
Oct 30 21:45:08.497775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.497704    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" podStartSLOduration=2.49768799 podStartE2EDuration="2.49768799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:08.496129062 +0000 UTC m=+2565.451816289" watchObservedRunningTime="2026-10-30 21:45:08.49768799 +0000 UTC m=+2565.453375218"
Oct 30 21:45:08.906823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:08.906790    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:09.747057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:09.747017    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:10.579392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:10.578386    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:11.394459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:11.391027    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:12.351810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:12.351769    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:13.326661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.326620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:13.769399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.769369    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:45:13.776067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.776044    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.805376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.805264    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-8072a0-openshift-default-dockercfg-2l5wp\""
Oct 30 21:45:13.810815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.810788    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:45:13.881852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.881821    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.881874    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882010    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pns6\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882053    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882116    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882174    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882258    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882340    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.882399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.882392    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.983930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.983899    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984000    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984073    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984128    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984222    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pns6\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984277    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984319    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984410    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984423    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984498    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.984934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984911    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.985024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.984939    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.985084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.985029    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.985370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.985344    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.987305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.987282    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.987421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.987401    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.994179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.994149    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:13.998737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:13.998713    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pns6\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\") pod \"e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") " pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:14.096127 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.096058    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:14.224238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.224193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:14.416209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.416175    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:14.416697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.416278    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:45:14.417212 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:45:14.417178    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc09f4b3_0436_41fa_ad66_86721bcee130.slice/crio-259cbb5f86f4d6452319075fb35d50c3ec19721b8d05c67a2c2b8e0d0eab93c1 WatchSource:0}: Error finding container 259cbb5f86f4d6452319075fb35d50c3ec19721b8d05c67a2c2b8e0d0eab93c1: Status 404 returned error can't find the container with id 259cbb5f86f4d6452319075fb35d50c3ec19721b8d05c67a2c2b8e0d0eab93c1
Oct 30 21:45:14.424094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.424044    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:45:14.424261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.424171    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:45:14.424261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.424244    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:45:14.512926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.512897    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" event={"ID":"bc09f4b3-0436-41fa-ad66-86721bcee130","Type":"ContainerStarted","Data":"259cbb5f86f4d6452319075fb35d50c3ec19721b8d05c67a2c2b8e0d0eab93c1"}
Oct 30 21:45:14.823279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:14.823187    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.142461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:15.138267    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.614514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:15.606416    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.942634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:15.910497    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:15.980161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:15.980127    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" event={"ID":"bc09f4b3-0436-41fa-ad66-86721bcee130","Type":"ContainerStarted","Data":"64db387be3c83cfab91aaec91fb2c21bdf5b7f83f87123884a901a0a62d64f5f"}
Oct 30 21:45:16.028609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.014782    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podStartSLOduration=3.01476456 podStartE2EDuration="3.01476456s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:16.012415894 +0000 UTC m=+2572.968103123" watchObservedRunningTime="2026-10-30 21:45:16.01476456 +0000 UTC m=+2572.970451787"
Oct 30 21:45:16.105911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.105877    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:16.106359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.106334    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 21:45:16.106556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.106529    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 21:45:16.157359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.157331    6738 scope.go:122] "RemoveContainer" containerID="4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"
Oct 30 21:45:16.157695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.157665    6738 scope.go:122] "RemoveContainer" containerID="f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"
Oct 30 21:45:16.157874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.157849    6738 scope.go:122] "RemoveContainer" containerID="dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"
Oct 30 21:45:16.625457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:16.621406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:17.053460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.046451    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerStarted","Data":"63bc8f9a90d2707c100c2a34d2891baf1a5c738a7b34513108afdef2fa2406fa"}
Oct 30 21:45:17.093780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.088077    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerStarted","Data":"98cd586d03ae9a801ffab1b973005225f8d6e240dc55cda30bdd082c86e5c670"}
Oct 30 21:45:17.108484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.107124    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 21:45:17.108484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.107176    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 21:45:17.149513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.135510    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerStarted","Data":"742c1cc6bce43652aae13c71d8984fcc40a36bb8cf60dea2ad5448026ab8776f"}
Oct 30 21:45:17.159115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:17.157290    6738 scope.go:122] "RemoveContainer" containerID="c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"
Oct 30 21:45:18.010459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.010018    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:18.109463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.103267    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 21:45:18.109463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.103331    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 21:45:18.123473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.120807    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:18.157057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.157016    6738 scope.go:122] "RemoveContainer" containerID="faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"
Oct 30 21:45:18.159176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.159144    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerStarted","Data":"1701fb377bd9caabd886fe16017dbdbdf151201fc9e2db2fd7080690df2ecb21"}
Oct 30 21:45:18.164741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.162647    6738 generic.go:356] "Generic (PLEG): container finished" podID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerID="63bc8f9a90d2707c100c2a34d2891baf1a5c738a7b34513108afdef2fa2406fa" exitCode=60
Oct 30 21:45:18.164741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.162734    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerDied","Data":"63bc8f9a90d2707c100c2a34d2891baf1a5c738a7b34513108afdef2fa2406fa"}
Oct 30 21:45:18.164741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.162772    6738 scope.go:122] "RemoveContainer" containerID="f4af590be049427389f56a75b8937db213572eaf79e20e515372cda2a0cce6a0"
Oct 30 21:45:18.165617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.165587    6738 generic.go:356] "Generic (PLEG): container finished" podID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerID="98cd586d03ae9a801ffab1b973005225f8d6e240dc55cda30bdd082c86e5c670" exitCode=60
Oct 30 21:45:18.165734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.165686    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerDied","Data":"98cd586d03ae9a801ffab1b973005225f8d6e240dc55cda30bdd082c86e5c670"}
Oct 30 21:45:18.166073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.166046    6738 scope.go:122] "RemoveContainer" containerID="98cd586d03ae9a801ffab1b973005225f8d6e240dc55cda30bdd082c86e5c670"
Oct 30 21:45:18.166354 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:18.166334    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8_odh-ai-gateway-infra(002dd75c-389b-4eb0-b1b7-9bc2ba42804c)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c"
Oct 30 21:45:18.168141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.168071    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:18.168885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.168864    6738 generic.go:356] "Generic (PLEG): container finished" podID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerID="742c1cc6bce43652aae13c71d8984fcc40a36bb8cf60dea2ad5448026ab8776f" exitCode=60
Oct 30 21:45:18.169179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.169150    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerDied","Data":"742c1cc6bce43652aae13c71d8984fcc40a36bb8cf60dea2ad5448026ab8776f"}
Oct 30 21:45:18.169759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.169740    6738 scope.go:122] "RemoveContainer" containerID="742c1cc6bce43652aae13c71d8984fcc40a36bb8cf60dea2ad5448026ab8776f"
Oct 30 21:45:18.169990 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:18.169969    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs_odh-ai-gateway-infra(daf6bc60-abf2-4196-926d-4bbbceeb099e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e"
Oct 30 21:45:18.255837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.255818    6738 scope.go:122] "RemoveContainer" containerID="dc0a4414368e92c88721c4d272470f48df2a434fba5c38fb6daadbeb6071dd56"
Oct 30 21:45:18.271039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.270996    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:18.286691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.286655    6738 scope.go:122] "RemoveContainer" containerID="4a172c79e040da6f611f3b1b2865745e5e07a6b43f9b3a7acf02658698ebb051"
Oct 30 21:45:18.471393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.471370    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:18.553775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.553745    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\") pod \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") "
Oct 30 21:45:18.553922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.553795    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\" (UniqueName: \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\") pod \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\" (UID: \"e6f803ab-d562-41e6-a257-ca8b1d1a310f\") "
Oct 30 21:45:18.554316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.554213    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca" pod "e6f803ab-d562-41e6-a257-ca8b1d1a310f" (UID: "e6f803ab-d562-41e6-a257-ca8b1d1a310f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:18.555177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.555147    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6f803ab-d562-41e6-a257-ca8b1d1a310f-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:18.557919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.557896    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq" pod "e6f803ab-d562-41e6-a257-ca8b1d1a310f" (UID: "e6f803ab-d562-41e6-a257-ca8b1d1a310f"). InnerVolumeSpecName "kube-api-access-cbgwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:18.656413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:18.656380    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbgwq\" (UniqueName: \"kubernetes.io/projected/e6f803ab-d562-41e6-a257-ca8b1d1a310f-kube-api-access-cbgwq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:19.111291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.111245    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 21:45:19.111934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.111889    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 21:45:19.285281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.285225    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:19.286411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.286381    6738 generic.go:356] "Generic (PLEG): container finished" podID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerID="1701fb377bd9caabd886fe16017dbdbdf151201fc9e2db2fd7080690df2ecb21" exitCode=60
Oct 30 21:45:19.317461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.311650    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:19.317461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.311755    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerDied","Data":"1701fb377bd9caabd886fe16017dbdbdf151201fc9e2db2fd7080690df2ecb21"}
Oct 30 21:45:19.317461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.311797    6738 scope.go:122] "RemoveContainer" containerID="c5d0ab93852c40e76292a409ef40d67bde86c1d30556b5d3e2eb2dc0edad7500"
Oct 30 21:45:19.361246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.360140    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerStarted","Data":"03e46514795e5fd2042fda3097f616ac05e68f5e10f151ad6bf343cb6531016d"}
Oct 30 21:45:19.367503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.367189    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"
Oct 30 21:45:19.367926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.367879    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f" event={"ID":"e6f803ab-d562-41e6-a257-ca8b1d1a310f","Type":"ContainerDied","Data":"b4f0057668cd9cab3dc6bd66a66522bb2513696ff4eae9a7f772e2d3c5efaa64"}
Oct 30 21:45:19.421486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.398707    6738 scope.go:122] "RemoveContainer" containerID="63bc8f9a90d2707c100c2a34d2891baf1a5c738a7b34513108afdef2fa2406fa"
Oct 30 21:45:19.462479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.459292    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:45:19.462479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.461669    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:45:19.557139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.557100    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:19.663623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.661532    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:19.668583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.668556    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-6ed261-29889945-hm82f"]
Oct 30 21:45:19.756457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.744478    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:19.867639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.859811    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\") pod \"daf6bc60-abf2-4196-926d-4bbbceeb099e\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") "
Oct 30 21:45:19.867639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.859968    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\" (UniqueName: \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\") pod \"daf6bc60-abf2-4196-926d-4bbbceeb099e\" (UID: \"daf6bc60-abf2-4196-926d-4bbbceeb099e\") "
Oct 30 21:45:19.867639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.861578    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca" pod "daf6bc60-abf2-4196-926d-4bbbceeb099e" (UID: "daf6bc60-abf2-4196-926d-4bbbceeb099e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:19.868949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.868406    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5" pod "daf6bc60-abf2-4196-926d-4bbbceeb099e" (UID: "daf6bc60-abf2-4196-926d-4bbbceeb099e"). InnerVolumeSpecName "kube-api-access-wfrq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:19.878880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.878686    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:19.972076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.971194    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/daf6bc60-abf2-4196-926d-4bbbceeb099e-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:19.972076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:19.971239    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfrq5\" (UniqueName: \"kubernetes.io/projected/daf6bc60-abf2-4196-926d-4bbbceeb099e-kube-api-access-wfrq5\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.032274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.031980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:20.034043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.033657    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:20.072567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.072537    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\") pod \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") "
Oct 30 21:45:20.072744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.072720    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\" (UniqueName: \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\") pod \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\" (UID: \"2e50bf6e-2a48-4f4c-abf5-0cf005499a81\") "
Oct 30 21:45:20.073695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.073671    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca" pod "2e50bf6e-2a48-4f4c-abf5-0cf005499a81" (UID: "2e50bf6e-2a48-4f4c-abf5-0cf005499a81"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:20.075987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.075960    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6" pod "2e50bf6e-2a48-4f4c-abf5-0cf005499a81" (UID: "2e50bf6e-2a48-4f4c-abf5-0cf005499a81"). InnerVolumeSpecName "kube-api-access-pbbn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:20.120080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.119599    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 21:45:20.120080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.119652    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 21:45:20.174614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.174578    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\" (UniqueName: \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\") pod \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") "
Oct 30 21:45:20.174846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.174831    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\") pod \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\" (UID: \"002dd75c-389b-4eb0-b1b7-9bc2ba42804c\") "
Oct 30 21:45:20.175216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.175192    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca" pod "002dd75c-389b-4eb0-b1b7-9bc2ba42804c" (UID: "002dd75c-389b-4eb0-b1b7-9bc2ba42804c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:20.175394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.175379    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.175484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.175397    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.175484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.175407    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbbn6\" (UniqueName: \"kubernetes.io/projected/2e50bf6e-2a48-4f4c-abf5-0cf005499a81-kube-api-access-pbbn6\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.176917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.176892    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d" pod "002dd75c-389b-4eb0-b1b7-9bc2ba42804c" (UID: "002dd75c-389b-4eb0-b1b7-9bc2ba42804c"). InnerVolumeSpecName "kube-api-access-cmn6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:20.276975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.276889    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmn6d\" (UniqueName: \"kubernetes.io/projected/002dd75c-389b-4eb0-b1b7-9bc2ba42804c-kube-api-access-cmn6d\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:20.361097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.361062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:20.383755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.383712    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"
Oct 30 21:45:20.383913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.383759    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8" event={"ID":"002dd75c-389b-4eb0-b1b7-9bc2ba42804c","Type":"ContainerDied","Data":"1e7e4264de1c90cf3a2914d5c1523798402a26e5d410b4e89f3ed75f07cdd4ee"}
Oct 30 21:45:20.383913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.383814    6738 scope.go:122] "RemoveContainer" containerID="98cd586d03ae9a801ffab1b973005225f8d6e240dc55cda30bdd082c86e5c670"
Oct 30 21:45:20.385352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.385326    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"
Oct 30 21:45:20.385514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.385354    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs" event={"ID":"daf6bc60-abf2-4196-926d-4bbbceeb099e","Type":"ContainerDied","Data":"ec1e3f88357a0c5de9c289e5d422af91f9f08813c71a78fd1c03506f451813df"}
Oct 30 21:45:20.386826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.386807    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"
Oct 30 21:45:20.386943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.386825    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd" event={"ID":"2e50bf6e-2a48-4f4c-abf5-0cf005499a81","Type":"ContainerDied","Data":"e42be9777db507c6b555f5e6913edd5804e622d314d30d2f0a2d9851ee59d45c"}
Oct 30 21:45:20.414685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.412901    6738 generic.go:356] "Generic (PLEG): container finished" podID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerID="03e46514795e5fd2042fda3097f616ac05e68f5e10f151ad6bf343cb6531016d" exitCode=60
Oct 30 21:45:20.414685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.412975    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerDied","Data":"03e46514795e5fd2042fda3097f616ac05e68f5e10f151ad6bf343cb6531016d"}
Oct 30 21:45:20.414685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.413396    6738 scope.go:122] "RemoveContainer" containerID="03e46514795e5fd2042fda3097f616ac05e68f5e10f151ad6bf343cb6531016d"
Oct 30 21:45:20.414685 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:20.413760    6738 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt_odh-ai-gateway-infra(3cbbcfd4-34ea-4c0d-8489-764092364dc4)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4"
Oct 30 21:45:20.430645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.427573    6738 scope.go:122] "RemoveContainer" containerID="742c1cc6bce43652aae13c71d8984fcc40a36bb8cf60dea2ad5448026ab8776f"
Oct 30 21:45:20.449759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.449735    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:20.454933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.454912    6738 scope.go:122] "RemoveContainer" containerID="1701fb377bd9caabd886fe16017dbdbdf151201fc9e2db2fd7080690df2ecb21"
Oct 30 21:45:20.471121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.471099    6738 scope.go:122] "RemoveContainer" containerID="faf82004b2ff8475128c1b27257ff156efd94c9327ad32815575b90c8bd9324b"
Oct 30 21:45:20.541499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.535028    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:20.545961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.545932    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-extmod-b-f1ed97-29889945-79sjs"]
Oct 30 21:45:20.551950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.551923    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:20.556581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.556559    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w1-d4a0a5-29889945-ptmmd"]
Oct 30 21:45:20.580559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.580535    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:20.644024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.643986    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-27804d-29889945-mzdh8"]
Oct 30 21:45:20.828549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:20.828466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:21.088508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.085428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:21.282456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.262108    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" path="/var/lib/kubelet/pods/002dd75c-389b-4eb0-b1b7-9bc2ba42804c/volumes"
Oct 30 21:45:21.282456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.263014    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" path="/var/lib/kubelet/pods/2e50bf6e-2a48-4f4c-abf5-0cf005499a81/volumes"
Oct 30 21:45:21.282456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.263767    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" path="/var/lib/kubelet/pods/daf6bc60-abf2-4196-926d-4bbbceeb099e/volumes"
Oct 30 21:45:21.282456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.278404    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" path="/var/lib/kubelet/pods/e6f803ab-d562-41e6-a257-ca8b1d1a310f/volumes"
Oct 30 21:45:21.386363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.386319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:21.581341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.579886    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:21.581908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.581869    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:21.588116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.586797    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:45:21.701235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.701206    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:21.840967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.840934    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\") pod \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") "
Oct 30 21:45:21.841140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.841001    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\" (UniqueName: \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\") pod \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\" (UID: \"3cbbcfd4-34ea-4c0d-8489-764092364dc4\") "
Oct 30 21:45:21.841357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.841331    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca" pod "3cbbcfd4-34ea-4c0d-8489-764092364dc4" (UID: "3cbbcfd4-34ea-4c0d-8489-764092364dc4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:45:21.841848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.841830    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cbbcfd4-34ea-4c0d-8489-764092364dc4-maas-api-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:21.843301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.843278    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k" pod "3cbbcfd4-34ea-4c0d-8489-764092364dc4" (UID: "3cbbcfd4-34ea-4c0d-8489-764092364dc4"). InnerVolumeSpecName "kube-api-access-rdz8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:45:21.943229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:21.943151    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdz8k\" (UniqueName: \"kubernetes.io/projected/3cbbcfd4-34ea-4c0d-8489-764092364dc4-kube-api-access-rdz8k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:45:22.460232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.460029    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:22.505678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.505650    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"
Oct 30 21:45:22.505879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.505654    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt" event={"ID":"3cbbcfd4-34ea-4c0d-8489-764092364dc4","Type":"ContainerDied","Data":"c5f4f39346a7accb4b6b126e8e119dddf132aa6d8cc9e1e584754e0b029d76dc"}
Oct 30 21:45:22.505879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.505781    6738 scope.go:122] "RemoveContainer" containerID="03e46514795e5fd2042fda3097f616ac05e68f5e10f151ad6bf343cb6531016d"
Oct 30 21:45:22.549059 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:22.548514    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:45:22.552054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.551493    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:22.556121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:22.556098    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-a33b12-29889945-9lhgt"]
Oct 30 21:45:23.178863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:23.177542    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" path="/var/lib/kubelet/pods/3cbbcfd4-34ea-4c0d-8489-764092364dc4/volumes"
Oct 30 21:45:23.626016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:23.625973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:24.436608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:24.436574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:25.205109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:25.205078    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:26.171344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:26.171293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:26.937117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:26.937081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:27.754071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:27.754040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:28.646774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:28.646717    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:29.699795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:29.699753    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:31.101460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:31.095797    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:32.581462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:32.572700    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:33.755458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:33.745852    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:34.626059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:34.626011    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:35.438919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:35.438889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:36.393765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:36.393081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:37.112761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:37.112729    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:38.038254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:38.038221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.059970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:39.059920    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:39.988319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:39.988281    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:40.846461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:40.846408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:41.666460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:41.666030    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:42.537340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:42.537310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:43.277240 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:43.277212    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:43.854479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:43.854412    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:44.098397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.098344    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.780405    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781625    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781642    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781659    6738 cpu_manager.go:414] "containerMap: removing container" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781668    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781681    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781689    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781743    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781752    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781792    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781799    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781811    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781818    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781845    6738 cpu_manager.go:414] "containerMap: removing container" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781852    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781863    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781869    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781882    6738 cpu_manager.go:414] "containerMap: removing container" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781888    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781897    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781904    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781919    6738 cpu_manager.go:414] "containerMap: removing container" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.781926    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782124    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782134    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782150    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782159    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782168    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782179    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782190    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782201    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782211    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782220    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782228    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782240    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:45:44.782813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.782248    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:45:44.798885 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.798857    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:44.806169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.806143    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:45:44.809990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.809965    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8431de-serving-cert\""
Oct 30 21:45:44.949955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.948504    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:44.951605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.951557    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r256s\" (UniqueName: \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:44.951872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.951850    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:44.952005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:44.951987    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.053714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.053634    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r256s\" (UniqueName: \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.053714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.053673    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.053909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.053744    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.053991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.053974    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.054537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.054511    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.056684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.056659    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.056794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.056666    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.057081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.057058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.070742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.070723    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r256s\" (UniqueName: \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\") pod \"maas-api-e2e-inf-a-8431de-575779cc9c-sfj44\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.126967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.126933    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:45.245480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.245311    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:45.468070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.468039    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:45:45.478788 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:45:45.478511    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc75253b_5d97_4da6_8fad_8fcd8245639a.slice/crio-31fd7de48e3805f8a90955df11157e6af98ac4fc30e1997008be90e7533f8682 WatchSource:0}: Error finding container 31fd7de48e3805f8a90955df11157e6af98ac4fc30e1997008be90e7533f8682: Status 404 returned error can't find the container with id 31fd7de48e3805f8a90955df11157e6af98ac4fc30e1997008be90e7533f8682
Oct 30 21:45:45.481554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.480421    6738 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 21:45:45.822808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:45.822719    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" event={"ID":"dc75253b-5d97-4da6-8fad-8fcd8245639a","Type":"ContainerStarted","Data":"31fd7de48e3805f8a90955df11157e6af98ac4fc30e1997008be90e7533f8682"}
Oct 30 21:45:46.194463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:46.187942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:46.833016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:46.832967    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" event={"ID":"dc75253b-5d97-4da6-8fad-8fcd8245639a","Type":"ContainerStarted","Data":"4f69656f4f3b7fbe9a6f6392779043bd6044a068b0b2dd3cfd6dae3f6d39837e"}
Oct 30 21:45:46.833245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:46.833224    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:46.869286 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:46.869240    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" podStartSLOduration=2.338006406 podStartE2EDuration="2.869226187s" totalImagesPullingTime="531.219781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:45:46.865967255 +0000 UTC m=+2603.821654481" watchObservedRunningTime="2026-10-30 21:45:46.869226187 +0000 UTC m=+2603.824913413"
Oct 30 21:45:47.144080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:47.144040    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:47.868619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:47.868586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:48.657062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:48.657014    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:49.413495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:49.413423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.218158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:50.218128    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:50.955349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:50.955322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:51.712667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:51.712626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:52.364087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:52.364052    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:52.512088 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:45:52.512044    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:45:52.845938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:52.845864    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:45:53.238687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:53.238656    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:53.951773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:53.951740    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:54.628502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:54.628467    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:55.431877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:55.431843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:56.282649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:56.282597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:56.927574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:56.927530    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:57.667550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:57.667519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:58.434338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:58.434304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:45:59.287255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:45:59.287224    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.031583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:00.031547    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:00.820635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:00.820598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:01.481891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:01.481853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:02.192969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:02.192934    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:03.031045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:03.031012    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:03.931997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:03.931963    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:04.739740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:04.739652    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:05.583520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:05.583488    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:06.305389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:06.305337    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:07.308686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:07.308654    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:08.133486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:08.133425    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:08.808167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:08.808127    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:09.648923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:09.648892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:10.432538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:10.432504    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:10.559171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:10.559143    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:11.009681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:11.009647    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:11.381323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:11.381289    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:11.505311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:11.505277    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:11.653482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:11.653378    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:12.446520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:12.446418    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:13.240341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:13.240315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:14.134330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:14.134294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:15.470731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:15.470699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:15.836331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:15.836244    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:15.960236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:15.960202    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:16.285206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:16.285170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:17.114536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:17.114489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:17.272598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:17.272558    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:18.135700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:18.135642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:18.894367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:18.894334    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:19.562941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:19.562898    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:20.286214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:20.286001    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:20.415869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:20.415841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:21.198279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:21.198218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:21.960067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:21.960037    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:22.514968 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:22.514921    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:46:22.651089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:22.651045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:23.450348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:23.450301    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:24.432224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:24.432191    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:25.142275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:25.142241    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:25.910610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:25.910575    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:26.822600 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:26.822564    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:27.469566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:27.469535    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:28.165775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:28.165730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:29.052206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:29.052166    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:29.729702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:29.729671    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:29.949864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:29.949830    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:30.067116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.067035    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:30.434098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434060    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:30.434905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434878    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:46:30.434905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434903    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3cbbcfd4-34ea-4c0d-8489-764092364dc4" containerName="cleanup"
Oct 30 21:46:30.434905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434919    6738 cpu_manager.go:414] "containerMap: removing container" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:46:30.434905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434925    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="002dd75c-389b-4eb0-b1b7-9bc2ba42804c" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434948    6738 cpu_manager.go:414] "containerMap: removing container" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434954    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434972    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.434976    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6f803ab-d562-41e6-a257-ca8b1d1a310f" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.435154    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e50bf6e-2a48-4f4c-abf5-0cf005499a81" containerName="cleanup"
Oct 30 21:46:30.435262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.435177    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="daf6bc60-abf2-4196-926d-4bbbceeb099e" containerName="cleanup"
Oct 30 21:46:30.439319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.439300    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.454275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.454251    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:30.673591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.673547    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbv8k\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.831992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.830654    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.831992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.830729    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.831992 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.831397    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.935156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.935119    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbv8k\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.935412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.935229    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.935412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.935264    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.935412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.935314    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.936141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.936111    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.936271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.936138    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.937984 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.937960    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:30.943952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:30.943928    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbv8k\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\") pod \"payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") " pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:31.057645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:31.057613    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:31.128815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:31.128779    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:31.349253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:31.349225    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:31.350305 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:46:31.350279    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff253c98_375d_450a_b66c_94a3146d3858.slice/crio-4a6b21b41920c82063441464cfb3f130f2ad023edaa71f55b2c349ae663b6f14 WatchSource:0}: Error finding container 4a6b21b41920c82063441464cfb3f130f2ad023edaa71f55b2c349ae663b6f14: Status 404 returned error can't find the container with id 4a6b21b41920c82063441464cfb3f130f2ad023edaa71f55b2c349ae663b6f14
Oct 30 21:46:31.909141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:31.909095    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:32.341732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:32.341631    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" event={"ID":"ff253c98-375d-450a-b66c-94a3146d3858","Type":"ContainerStarted","Data":"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"}
Oct 30 21:46:32.341732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:32.341698    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" event={"ID":"ff253c98-375d-450a-b66c-94a3146d3858","Type":"ContainerStarted","Data":"4a6b21b41920c82063441464cfb3f130f2ad023edaa71f55b2c349ae663b6f14"}
Oct 30 21:46:32.341732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:32.341725    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:32.361923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:32.361868    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" podStartSLOduration=2.361850996 podStartE2EDuration="2.361850996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:32.358242131 +0000 UTC m=+2649.313929363" watchObservedRunningTime="2026-10-30 21:46:32.361850996 +0000 UTC m=+2649.317538250"
Oct 30 21:46:32.581247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:32.577050    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:33.440569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:33.440537    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:33.991077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:33.991036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:34.147764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.147733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:34.216556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.216512    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:34.238495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.232361    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.241640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.241475    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-5ac5f2767576\""
Oct 30 21:46:34.242073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.242038    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-jgk7c\""
Oct 30 21:46:34.244766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.244734    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 21:46:34.245037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.245021    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 21:46:34.245230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.245216    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 21:46:34.245453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.245420    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 21:46:34.250531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.250505    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:34.291485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.289366    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.291485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.289430    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.291485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.289583    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.291485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.289802    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmv4k\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.391040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.391004    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmv4k\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.391233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.391169    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.391307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.391228    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.391307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.391299    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.392106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.392081    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.392223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.392113    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.393999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.393977    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.399488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.399460    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmv4k\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\") pod \"payload-processing-external-model-5947f4d466-qb4q7\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") " pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.583103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.582732    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:34.975341 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:46:34.975308    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233b9f87_d3b5_4daa_8648_c22af97a6922.slice/crio-ed2b550c6ae333f421d198cb30779b7509405d32022a195769dce04feb0c15c8 WatchSource:0}: Error finding container ed2b550c6ae333f421d198cb30779b7509405d32022a195769dce04feb0c15c8: Status 404 returned error can't find the container with id ed2b550c6ae333f421d198cb30779b7509405d32022a195769dce04feb0c15c8
Oct 30 21:46:34.975494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:34.975405    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:35.122338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.122304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:35.370408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.370372    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" event={"ID":"233b9f87-d3b5-4daa-8648-c22af97a6922","Type":"ContainerStarted","Data":"b89fe6acab9119765a2c940d2704d3e09631bad269cf254d48adf96f1dde674a"}
Oct 30 21:46:35.370626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.370456    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" event={"ID":"233b9f87-d3b5-4daa-8648-c22af97a6922","Type":"ContainerStarted","Data":"ed2b550c6ae333f421d198cb30779b7509405d32022a195769dce04feb0c15c8"}
Oct 30 21:46:35.370626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.370483    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:35.394938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.394887    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" podStartSLOduration=1.394872274 podStartE2EDuration="1.394872274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:35.391797679 +0000 UTC m=+2652.347484908" watchObservedRunningTime="2026-10-30 21:46:35.394872274 +0000 UTC m=+2652.350559523"
Oct 30 21:46:35.645857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.645766    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:35.685457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.683058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:35.815474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:35.808889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:36.203672 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:36.203636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:36.379857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:36.379821    6738 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" secret="" err="secret \"payload-processing-external-model-dockercfg-jgk7c\" not found"
Oct 30 21:46:36.477557 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.474029    6738 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-5ac5f2767576: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:36.477557 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.474057    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:36.477557 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.474113    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:36.974092268 +0000 UTC m=+2653.929779502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:36.477557 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.474582    6738 configmap.go:193] Couldn't get configMap models-as-a-service/payload-processing-external-model-plugins: configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:36.477557 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.474630    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:36.974618104 +0000 UTC m=+2653.930305310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:36.573238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:36.573200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:36.756511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:36.753700    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:36.991412 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.991378    6738 configmap.go:193] Couldn't get configMap models-as-a-service/payload-processing-external-model-plugins: configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:36.991792 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.991773    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:37.99174845 +0000 UTC m=+2654.947435670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:36.991940 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.991829    6738 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-5ac5f2767576: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:36.992298 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.992277    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:36.992534 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:36.992518    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:37.992502595 +0000 UTC m=+2654.948189815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:37.577808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:37.577773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:38.019461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:38.014023    6738 configmap.go:193] Couldn't get configMap models-as-a-service/payload-processing-external-model-plugins: configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:38.019461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:38.014109    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:40.014089698 +0000 UTC m=+2656.969776907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "plugins-config-volume" (UniqueName: "kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : configmap "payload-processing-external-model-plugins" not found
Oct 30 21:46:38.019461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:38.014584    6738 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-5ac5f2767576: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:38.019461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:38.014605    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:38.019461 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:38.014655    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:40.014639247 +0000 UTC m=+2656.970326470 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:38.080960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:38.080923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:38.189465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:38.187072    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:39.281470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.278598    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:39.342006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.339735    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:39.365487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.357763    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.365487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.362904    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:39.365487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.363160    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-d5936ba6bf73\""
Oct 30 21:46:39.365487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.363376    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-bnw5z\""
Oct 30 21:46:39.408668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.408635    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.409150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.409122    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.409494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.409422    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs4v5\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.409647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.409631    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.417248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.417225    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:39.516459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.514578    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.516459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.514748    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs4v5\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.516459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.514793    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.516459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.514886    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.516459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.516068    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.520533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.517052    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.551214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.548148    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs4v5\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.583460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.579116    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-xfn5c\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:39.687460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:39.687110    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:40.032000 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:40.031966    6738 projected.go:266] Couldn't get secret models-as-a-service/agc-cred-5ac5f2767576: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:40.032000 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:40.031990    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7: secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:40.032254 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:40.032050    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials podName:233b9f87-d3b5-4daa-8648-c22af97a6922 nodeName:}" failed. No retries permitted until 2026-10-30 21:46:44.032032222 +0000 UTC m=+2660.987719447 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials") pod "payload-processing-external-model-5947f4d466-qb4q7" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922") : secret "agc-cred-5ac5f2767576" not found
Oct 30 21:46:40.088342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.088314    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:40.091311 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:46:40.091286    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239b7caa_35c7_4789_a858_44ae030976ee.slice/crio-2ed8c53aa2fdd1a5643a52ac27fdbc02658ac90139a77e6da82bd573619fb644 WatchSource:0}: Error finding container 2ed8c53aa2fdd1a5643a52ac27fdbc02658ac90139a77e6da82bd573619fb644: Status 404 returned error can't find the container with id 2ed8c53aa2fdd1a5643a52ac27fdbc02658ac90139a77e6da82bd573619fb644
Oct 30 21:46:40.450677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.450582    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:46:40.456396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.456370    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.462914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.462871    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-570f01-serving-cert\""
Oct 30 21:46:40.466180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.466147    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:46:40.536403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.536258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:40.538412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.538304    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" event={"ID":"239b7caa-35c7-4789-a858-44ae030976ee","Type":"ContainerStarted","Data":"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"}
Oct 30 21:46:40.538766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.538634    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" event={"ID":"239b7caa-35c7-4789-a858-44ae030976ee","Type":"ContainerStarted","Data":"2ed8c53aa2fdd1a5643a52ac27fdbc02658ac90139a77e6da82bd573619fb644"}
Oct 30 21:46:40.539726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.539553    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:40.541316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.541210    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.541637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.541614    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85bnq\" (UniqueName: \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.541869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.541848    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.542074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.542055    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.567482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.567386    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" podStartSLOduration=1.5673677019999999 podStartE2EDuration="1.567367702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:40.559161292 +0000 UTC m=+2657.514848526" watchObservedRunningTime="2026-10-30 21:46:40.567367702 +0000 UTC m=+2657.523054937"
Oct 30 21:46:40.643333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.643293    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:40.643793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.643766    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85bnq\" (UniqueName: \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.643902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.643822    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.643902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.643893    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.644164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.644148    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.645522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.645492    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.647365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.647340    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.649166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.649147    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.653508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.653483    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85bnq\" (UniqueName: \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\") pod \"maas-api-e2e-inf-b-570f01-686c544888-nkp7k\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.775593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.772002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:40.785064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.783770    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:40.934717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:40.934653    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:41.083665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:41.083575    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:41.137430 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:41.137399    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:46:41.137618 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:46:41.137502    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c81aeb_8918_4fb7_8e4e_0cce70d6c11e.slice/crio-926e8d69f20b3bfab140d8dae41f1ea02fd7668f155f2ed1a8519393bf1456cf WatchSource:0}: Error finding container 926e8d69f20b3bfab140d8dae41f1ea02fd7668f155f2ed1a8519393bf1456cf: Status 404 returned error can't find the container with id 926e8d69f20b3bfab140d8dae41f1ea02fd7668f155f2ed1a8519393bf1456cf
Oct 30 21:46:41.583899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:41.583801    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" event={"ID":"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e","Type":"ContainerStarted","Data":"926e8d69f20b3bfab140d8dae41f1ea02fd7668f155f2ed1a8519393bf1456cf"}
Oct 30 21:46:42.033249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.033218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:42.215891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.215857    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:42.398467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.394796    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" podUID="233b9f87-d3b5-4daa-8648-c22af97a6922" containerName="payload-processing" containerID="cri-o://b89fe6acab9119765a2c940d2704d3e09631bad269cf254d48adf96f1dde674a" gracePeriod=25
Oct 30 21:46:42.410492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.410392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:42.663420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.663366    6738 generic.go:356] "Generic (PLEG): container finished" podID="233b9f87-d3b5-4daa-8648-c22af97a6922" containerID="b89fe6acab9119765a2c940d2704d3e09631bad269cf254d48adf96f1dde674a" exitCode=0
Oct 30 21:46:42.664301 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.664271    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" event={"ID":"233b9f87-d3b5-4daa-8648-c22af97a6922","Type":"ContainerDied","Data":"b89fe6acab9119765a2c940d2704d3e09631bad269cf254d48adf96f1dde674a"}
Oct 30 21:46:42.669070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.666998    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" event={"ID":"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e","Type":"ContainerStarted","Data":"37e093a829d6aee28aafc9fce0694f48d5bcc94a5bb57d7ba731d53fec171c00"}
Oct 30 21:46:42.670147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.670086    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:42.696583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.696530    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" podStartSLOduration=2.149844856 podStartE2EDuration="2.696514468s" totalImagesPullingTime="546.669612ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:42.691087324 +0000 UTC m=+2659.646774553" watchObservedRunningTime="2026-10-30 21:46:42.696514468 +0000 UTC m=+2659.652201695"
Oct 30 21:46:42.960822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:42.960797    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:43.044895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.044861    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\") pod \"233b9f87-d3b5-4daa-8648-c22af97a6922\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") "
Oct 30 21:46:43.045107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.044981    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\") pod \"233b9f87-d3b5-4daa-8648-c22af97a6922\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") "
Oct 30 21:46:43.045269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.045249    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\") pod \"233b9f87-d3b5-4daa-8648-c22af97a6922\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") "
Oct 30 21:46:43.045343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.045301    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\") pod \"233b9f87-d3b5-4daa-8648-c22af97a6922\" (UID: \"233b9f87-d3b5-4daa-8648-c22af97a6922\") "
Oct 30 21:46:43.045415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.045388    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume" pod "233b9f87-d3b5-4daa-8648-c22af97a6922" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:43.045918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.045893    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume" pod "233b9f87-d3b5-4daa-8648-c22af97a6922" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:43.046110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.046088    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:43.046195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.046117    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/233b9f87-d3b5-4daa-8648-c22af97a6922-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:43.049702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.049673    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k" pod "233b9f87-d3b5-4daa-8648-c22af97a6922" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922"). InnerVolumeSpecName "kube-api-access-nmv4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:43.050541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.050512    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials" pod "233b9f87-d3b5-4daa-8648-c22af97a6922" (UID: "233b9f87-d3b5-4daa-8648-c22af97a6922"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:43.147395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.147361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:43.147890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.147861    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:43.147985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.147897    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmv4k\" (UniqueName: \"kubernetes.io/projected/233b9f87-d3b5-4daa-8648-c22af97a6922-kube-api-access-nmv4k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:43.355133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.355102    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:43.416012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.415981    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:46:43.677351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.677324    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"
Oct 30 21:46:43.677905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.677365    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7" event={"ID":"233b9f87-d3b5-4daa-8648-c22af97a6922","Type":"ContainerDied","Data":"ed2b550c6ae333f421d198cb30779b7509405d32022a195769dce04feb0c15c8"}
Oct 30 21:46:43.677905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.677421    6738 scope.go:122] "RemoveContainer" containerID="b89fe6acab9119765a2c940d2704d3e09631bad269cf254d48adf96f1dde674a"
Oct 30 21:46:43.718045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.717570    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:43.722189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:43.722163    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-5947f4d466-qb4q7"]
Oct 30 21:46:44.005019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:44.004916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:44.159083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:44.155408    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:44.225479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:44.225423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:44.425458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:44.413184    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:45.225462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:45.214828    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:45.258464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:45.241575    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="233b9f87-d3b5-4daa-8648-c22af97a6922" path="/var/lib/kubelet/pods/233b9f87-d3b5-4daa-8648-c22af97a6922/volumes"
Oct 30 21:46:45.384557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:45.383338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:45.550571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:45.550381    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.266532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.266499    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.334105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.334067    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:46.335389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.335368    6738 cpu_manager.go:414] "containerMap: removing container" podUID="233b9f87-d3b5-4daa-8648-c22af97a6922" containerName="payload-processing"
Oct 30 21:46:46.335461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.335393    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="233b9f87-d3b5-4daa-8648-c22af97a6922" containerName="payload-processing"
Oct 30 21:46:46.335702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.335680    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="233b9f87-d3b5-4daa-8648-c22af97a6922" containerName="payload-processing"
Oct 30 21:46:46.341828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.341807    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.353531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.353504    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-2ljkr\""
Oct 30 21:46:46.358793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.358766    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:46.477545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.477517    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:46.488408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.488372    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.488608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.488575    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.488731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.488701    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.488830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.488778    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rnlj\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.593515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.592609    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.593515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.592837    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.593515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.592915    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.593515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.592970    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rnlj\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.597456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.594997    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.597456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.595891    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.628460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.628149    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.660774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.660705    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rnlj\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\") pod \"payload-processing-external-model-575646b4c4-mgxhp\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") " pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:46.679462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:46.664080    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:47.150031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.150002    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:47.150232 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:46:47.150207    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda22b2543_d951_44e4_bbd9_7bc2f48eaf3b.slice/crio-803e1d9bf70d71fe2c812cc4f4eddb86c423e13dcbdbd2c6b87ebd6a66ff7406 WatchSource:0}: Error finding container 803e1d9bf70d71fe2c812cc4f4eddb86c423e13dcbdbd2c6b87ebd6a66ff7406: Status 404 returned error can't find the container with id 803e1d9bf70d71fe2c812cc4f4eddb86c423e13dcbdbd2c6b87ebd6a66ff7406
Oct 30 21:46:47.295462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.290945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:47.843976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.843937    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" event={"ID":"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b","Type":"ContainerStarted","Data":"4174bf544abbd8e37ccb4510ace031d8be7a9f0fd283c43ec9c748182cf8a9c7"}
Oct 30 21:46:47.844204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.844029    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:47.844204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.844052    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" event={"ID":"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b","Type":"ContainerStarted","Data":"803e1d9bf70d71fe2c812cc4f4eddb86c423e13dcbdbd2c6b87ebd6a66ff7406"}
Oct 30 21:46:47.867310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.867216    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" podStartSLOduration=1.867201766 podStartE2EDuration="1.867201766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:46:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:46:47.864877053 +0000 UTC m=+2664.820564309" watchObservedRunningTime="2026-10-30 21:46:47.867201766 +0000 UTC m=+2664.822888994"
Oct 30 21:46:47.971417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:47.971372    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:48.417087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.417026    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" podUID="74ee710b-c77f-4452-a567-bd5ff944fac4" containerName="payload-processing" containerID="cri-o://ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629" gracePeriod=25
Oct 30 21:46:48.774987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.774957    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:46:48.820175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.820138    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:48.858480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.858404    6738 generic.go:356] "Generic (PLEG): container finished" podID="74ee710b-c77f-4452-a567-bd5ff944fac4" containerID="ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629" exitCode=0
Oct 30 21:46:48.858675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.858473    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" event={"ID":"74ee710b-c77f-4452-a567-bd5ff944fac4","Type":"ContainerDied","Data":"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"}
Oct 30 21:46:48.858675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.858578    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz" event={"ID":"74ee710b-c77f-4452-a567-bd5ff944fac4","Type":"ContainerDied","Data":"4a26b85cdecd48d0cc8cf3320f55d9d8ba8fac4dc923ca6788cd5ccbc35db52e"}
Oct 30 21:46:48.858675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.858613    6738 scope.go:122] "RemoveContainer" containerID="ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"
Oct 30 21:46:48.858935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.858914    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"
Oct 30 21:46:48.874557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.874532    6738 scope.go:122] "RemoveContainer" containerID="ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"
Oct 30 21:46:48.874992 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:48.874970    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629\": container with ID starting with ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629 not found: ID does not exist" containerID="ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"
Oct 30 21:46:48.875078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.875005    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629"} err="failed to get container status \"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629\": rpc error: code = NotFound desc = could not find container \"ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629\": container with ID starting with ffcd50a37565159ce777be6f01953c53c42fdd153e4c77b5574222eaf1109629 not found: ID does not exist"
Oct 30 21:46:48.880612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.880583    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\") pod \"74ee710b-c77f-4452-a567-bd5ff944fac4\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") "
Oct 30 21:46:48.880894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.880871    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\") pod \"74ee710b-c77f-4452-a567-bd5ff944fac4\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") "
Oct 30 21:46:48.881055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.881037    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\") pod \"74ee710b-c77f-4452-a567-bd5ff944fac4\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") "
Oct 30 21:46:48.881169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.881153    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\") pod \"74ee710b-c77f-4452-a567-bd5ff944fac4\" (UID: \"74ee710b-c77f-4452-a567-bd5ff944fac4\") "
Oct 30 21:46:48.881462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.881411    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume" pod "74ee710b-c77f-4452-a567-bd5ff944fac4" (UID: "74ee710b-c77f-4452-a567-bd5ff944fac4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:48.881553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.881429    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume" pod "74ee710b-c77f-4452-a567-bd5ff944fac4" (UID: "74ee710b-c77f-4452-a567-bd5ff944fac4"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:48.882691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.882664    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:48.882764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.882696    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74ee710b-c77f-4452-a567-bd5ff944fac4-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:48.883106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.883086    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz" pod "74ee710b-c77f-4452-a567-bd5ff944fac4" (UID: "74ee710b-c77f-4452-a567-bd5ff944fac4"). InnerVolumeSpecName "kube-api-access-kt2wz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:48.883771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.883748    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials" pod "74ee710b-c77f-4452-a567-bd5ff944fac4" (UID: "74ee710b-c77f-4452-a567-bd5ff944fac4"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:48.985771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.984615    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt2wz\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-kube-api-access-kt2wz\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:48.985771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:48.984650    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/74ee710b-c77f-4452-a567-bd5ff944fac4-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:49.156309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.156258    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" podUID="239b7caa-35c7-4789-a858-44ae030976ee" containerName="payload-processing" containerID="cri-o://e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01" gracePeriod=25
Oct 30 21:46:49.195052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.195011    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:46:49.198934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.198906    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-5d4fqz2zz"]
Oct 30 21:46:49.450422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.450399    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:49.495349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.494315    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\") pod \"239b7caa-35c7-4789-a858-44ae030976ee\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") "
Oct 30 21:46:49.495349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.494978    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\") pod \"239b7caa-35c7-4789-a858-44ae030976ee\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") "
Oct 30 21:46:49.495349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.495104    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\") pod \"239b7caa-35c7-4789-a858-44ae030976ee\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") "
Oct 30 21:46:49.495349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.495206    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\") pod \"239b7caa-35c7-4789-a858-44ae030976ee\" (UID: \"239b7caa-35c7-4789-a858-44ae030976ee\") "
Oct 30 21:46:49.496767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.496187    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume" pod "239b7caa-35c7-4789-a858-44ae030976ee" (UID: "239b7caa-35c7-4789-a858-44ae030976ee"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:49.496767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.496689    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume" pod "239b7caa-35c7-4789-a858-44ae030976ee" (UID: "239b7caa-35c7-4789-a858-44ae030976ee"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:49.497362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.497335    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:49.497496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.497366    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/239b7caa-35c7-4789-a858-44ae030976ee-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:49.499955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.499927    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials" pod "239b7caa-35c7-4789-a858-44ae030976ee" (UID: "239b7caa-35c7-4789-a858-44ae030976ee"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:49.515541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.503155    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5" pod "239b7caa-35c7-4789-a858-44ae030976ee" (UID: "239b7caa-35c7-4789-a858-44ae030976ee"). InnerVolumeSpecName "kube-api-access-qs4v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:49.599214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.599132    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs4v5\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-kube-api-access-qs4v5\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:49.599214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.599162    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/239b7caa-35c7-4789-a858-44ae030976ee-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:49.640540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.640493    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:49.753511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.749952    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:46:49.869758 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.869727    6738 generic.go:356] "Generic (PLEG): container finished" podID="239b7caa-35c7-4789-a858-44ae030976ee" containerID="e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01" exitCode=0
Oct 30 21:46:49.870058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.870041    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"
Oct 30 21:46:49.870567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.870519    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" event={"ID":"239b7caa-35c7-4789-a858-44ae030976ee","Type":"ContainerDied","Data":"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"}
Oct 30 21:46:49.870693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.870600    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c" event={"ID":"239b7caa-35c7-4789-a858-44ae030976ee","Type":"ContainerDied","Data":"2ed8c53aa2fdd1a5643a52ac27fdbc02658ac90139a77e6da82bd573619fb644"}
Oct 30 21:46:49.870693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.870635    6738 scope.go:122] "RemoveContainer" containerID="e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"
Oct 30 21:46:49.882330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.882308    6738 scope.go:122] "RemoveContainer" containerID="e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"
Oct 30 21:46:49.882785 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:49.882670    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01\": container with ID starting with e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01 not found: ID does not exist" containerID="e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"
Oct 30 21:46:49.882785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.882713    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01"} err="failed to get container status \"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01\": rpc error: code = NotFound desc = could not find container \"e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01\": container with ID starting with e551214d65ad690ef86ec4a0afaccce40b8ec565f8380d31af6ee42347222b01 not found: ID does not exist"
Oct 30 21:46:49.905876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.905846    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:49.909653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:49.909629    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-xfn5c"]
Oct 30 21:46:50.434176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:50.434144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:51.170400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.168216    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="239b7caa-35c7-4789-a858-44ae030976ee" path="/var/lib/kubelet/pods/239b7caa-35c7-4789-a858-44ae030976ee/volumes"
Oct 30 21:46:51.170400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.169032    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74ee710b-c77f-4452-a567-bd5ff944fac4" path="/var/lib/kubelet/pods/74ee710b-c77f-4452-a567-bd5ff944fac4/volumes"
Oct 30 21:46:51.227693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.227652    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:51.748661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.748625    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:51.786662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.786626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:51.929466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:51.929190    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:52.407214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:52.407175    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:52.531128 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:52.531095    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:46:52.730514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:52.729645    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:52.915099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:52.915059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:53.393483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:53.393418    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:53.411489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:53.411367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:53.531588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:53.531552    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:54.233937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:54.233904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:55.044159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:55.044104    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:55.751081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:55.751022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:56.500611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:56.500492    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:56.748975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:56.748935    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" podUID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" containerName="payload-processing" containerID="cri-o://4174bf544abbd8e37ccb4510ace031d8be7a9f0fd283c43ec9c748182cf8a9c7" gracePeriod=25
Oct 30 21:46:56.973904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:56.973865    6738 generic.go:356] "Generic (PLEG): container finished" podID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" containerID="4174bf544abbd8e37ccb4510ace031d8be7a9f0fd283c43ec9c748182cf8a9c7" exitCode=0
Oct 30 21:46:56.974110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:56.973946    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" event={"ID":"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b","Type":"ContainerDied","Data":"4174bf544abbd8e37ccb4510ace031d8be7a9f0fd283c43ec9c748182cf8a9c7"}
Oct 30 21:46:57.066784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.066751    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:57.128645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.128605    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\") pod \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") "
Oct 30 21:46:57.128851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.128787    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\") pod \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") "
Oct 30 21:46:57.128851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.128839    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\") pod \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") "
Oct 30 21:46:57.128981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.128933    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\") pod \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\" (UID: \"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b\") "
Oct 30 21:46:57.129358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.129326    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume" pod "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" (UID: "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:57.129689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.129390    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume" pod "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" (UID: "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:57.129795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.129774    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:57.129863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.129800    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:57.134812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.131327    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials" pod "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" (UID: "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:57.134812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.131657    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj" pod "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" (UID: "a22b2543-d951-44e4-bbd9-7bc2f48eaf3b"). InnerVolumeSpecName "kube-api-access-6rnlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:57.232385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.232271    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:57.232385 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.232309    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rnlj\" (UniqueName: \"kubernetes.io/projected/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b-kube-api-access-6rnlj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:57.295121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.295030    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:57.985281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.985241    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp" event={"ID":"a22b2543-d951-44e4-bbd9-7bc2f48eaf3b","Type":"ContainerDied","Data":"803e1d9bf70d71fe2c812cc4f4eddb86c423e13dcbdbd2c6b87ebd6a66ff7406"}
Oct 30 21:46:57.985826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.985291    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"
Oct 30 21:46:57.985826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:57.985353    6738 scope.go:122] "RemoveContainer" containerID="4174bf544abbd8e37ccb4510ace031d8be7a9f0fd283c43ec9c748182cf8a9c7"
Oct 30 21:46:58.017801 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.017766    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:58.024145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.024111    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-575646b4c4-mgxhp"]
Oct 30 21:46:58.150073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.150038    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:58.394587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.394541    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" podUID="ff253c98-375d-450a-b66c-94a3146d3858" containerName="payload-processing" containerID="cri-o://5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597" gracePeriod=25
Oct 30 21:46:58.726170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.726138    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:58.765623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.765590    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\") pod \"ff253c98-375d-450a-b66c-94a3146d3858\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") "
Oct 30 21:46:58.765823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.765704    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\") pod \"ff253c98-375d-450a-b66c-94a3146d3858\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") "
Oct 30 21:46:58.765904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.765890    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\") pod \"ff253c98-375d-450a-b66c-94a3146d3858\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") "
Oct 30 21:46:58.765965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.765951    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\") pod \"ff253c98-375d-450a-b66c-94a3146d3858\" (UID: \"ff253c98-375d-450a-b66c-94a3146d3858\") "
Oct 30 21:46:58.766357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.766025    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume" pod "ff253c98-375d-450a-b66c-94a3146d3858" (UID: "ff253c98-375d-450a-b66c-94a3146d3858"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:58.766678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.766571    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume" pod "ff253c98-375d-450a-b66c-94a3146d3858" (UID: "ff253c98-375d-450a-b66c-94a3146d3858"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:46:58.766940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.766733    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.766940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.766757    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/ff253c98-375d-450a-b66c-94a3146d3858-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.768820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.768795    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k" pod "ff253c98-375d-450a-b66c-94a3146d3858" (UID: "ff253c98-375d-450a-b66c-94a3146d3858"). InnerVolumeSpecName "kube-api-access-fbv8k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:58.771627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.771601    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials" pod "ff253c98-375d-450a-b66c-94a3146d3858" (UID: "ff253c98-375d-450a-b66c-94a3146d3858"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:46:58.868256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.868216    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbv8k\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-kube-api-access-fbv8k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.868256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.868251    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/ff253c98-375d-450a-b66c-94a3146d3858-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:46:58.997139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.997037    6738 generic.go:356] "Generic (PLEG): container finished" podID="ff253c98-375d-450a-b66c-94a3146d3858" containerID="5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597" exitCode=0
Oct 30 21:46:58.997139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.997097    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"
Oct 30 21:46:58.997721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.997138    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" event={"ID":"ff253c98-375d-450a-b66c-94a3146d3858","Type":"ContainerDied","Data":"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"}
Oct 30 21:46:58.997721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.997204    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j" event={"ID":"ff253c98-375d-450a-b66c-94a3146d3858","Type":"ContainerDied","Data":"4a6b21b41920c82063441464cfb3f130f2ad023edaa71f55b2c349ae663b6f14"}
Oct 30 21:46:58.997721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:58.997234    6738 scope.go:122] "RemoveContainer" containerID="5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"
Oct 30 21:46:59.020424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.020170    6738 scope.go:122] "RemoveContainer" containerID="5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"
Oct 30 21:46:59.020566 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:46:59.020525    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597\": container with ID starting with 5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597 not found: ID does not exist" containerID="5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"
Oct 30 21:46:59.020641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.020561    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597"} err="failed to get container status \"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597\": rpc error: code = NotFound desc = could not find container \"5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597\": container with ID starting with 5483220cb7bdfbf32d888b28c2448c3be921e512910dcf9a4f0e49352238c597 not found: ID does not exist"
Oct 30 21:46:59.034420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.034388    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:59.042627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.042605    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-36a099/payload-processing-external-model-e2e-extmod-b-36a099-67c7n567j"]
Oct 30 21:46:59.062626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.062599    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:46:59.167821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.167743    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" path="/var/lib/kubelet/pods/a22b2543-d951-44e4-bbd9-7bc2f48eaf3b/volumes"
Oct 30 21:46:59.168307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.168268    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff253c98-375d-450a-b66c-94a3146d3858" path="/var/lib/kubelet/pods/ff253c98-375d-450a-b66c-94a3146d3858/volumes"
Oct 30 21:46:59.937093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:46:59.937062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:00.283919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:00.283821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:00.399755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:00.399482    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:00.782421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:00.782384    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:00.926747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:00.915946    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:01.326010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.325972    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:47:01.326892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326870    6738 cpu_manager.go:414] "containerMap: removing container" podUID="239b7caa-35c7-4789-a858-44ae030976ee" containerName="payload-processing"
Oct 30 21:47:01.326892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326888    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="239b7caa-35c7-4789-a858-44ae030976ee" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326903    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326912    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326935    6738 cpu_manager.go:414] "containerMap: removing container" podUID="74ee710b-c77f-4452-a567-bd5ff944fac4" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326941    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74ee710b-c77f-4452-a567-bd5ff944fac4" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326958    6738 cpu_manager.go:414] "containerMap: removing container" podUID="ff253c98-375d-450a-b66c-94a3146d3858" containerName="payload-processing"
Oct 30 21:47:01.327067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.326963    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff253c98-375d-450a-b66c-94a3146d3858" containerName="payload-processing"
Oct 30 21:47:01.327480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.327100    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="74ee710b-c77f-4452-a567-bd5ff944fac4" containerName="payload-processing"
Oct 30 21:47:01.327480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.327114    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff253c98-375d-450a-b66c-94a3146d3858" containerName="payload-processing"
Oct 30 21:47:01.327480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.327126    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="239b7caa-35c7-4789-a858-44ae030976ee" containerName="payload-processing"
Oct 30 21:47:01.327480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.327134    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a22b2543-d951-44e4-bbd9-7bc2f48eaf3b" containerName="payload-processing"
Oct 30 21:47:01.335840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.335791    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.342872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.342844    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-3d9d7b-openshift-default-dockercfg-n9f2r\""
Oct 30 21:47:01.360303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.360275    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445605    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445687    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445768    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445830    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445868    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.445928    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.446081    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.446127    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.451459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.446219    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbjss\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.485566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.482032    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:01.560461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.558500    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.560461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.558568    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.560461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.558659    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjss\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.560461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.558755    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.560461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.558834    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.603510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.602479    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.603510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.602750    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.604559    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.604630    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.604673    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.604746    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.605228    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.608458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.605876    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.633088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.617408    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.633722 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.633664    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.663457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.663212    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjss\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.667487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.663702    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.667487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.664287    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\") pod \"e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") " pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:01.963469 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:01.963396    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:02.074053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.074015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.256746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.256663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:02.365721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.365663    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:47:02.369376 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:47:02.369345    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod817ed678_747a_494d_979e_10c0170ef7ca.slice/crio-99451d54a9a826f743fc60af8774b7d2ee9467cf222f3254d6e723698ac30447 WatchSource:0}: Error finding container 99451d54a9a826f743fc60af8774b7d2ee9467cf222f3254d6e723698ac30447: Status 404 returned error can't find the container with id 99451d54a9a826f743fc60af8774b7d2ee9467cf222f3254d6e723698ac30447
Oct 30 21:47:02.372661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.372624    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:02.372782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.372709    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:02.372782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:02.372755    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:03.067044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.067002    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerStarted","Data":"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"}
Oct 30 21:47:03.067216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.067068    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerStarted","Data":"99451d54a9a826f743fc60af8774b7d2ee9467cf222f3254d6e723698ac30447"}
Oct 30 21:47:03.098626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.098568    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podStartSLOduration=2.098549933 podStartE2EDuration="2.098549933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:03.089061725 +0000 UTC m=+2680.044748952" watchObservedRunningTime="2026-10-30 21:47:03.098549933 +0000 UTC m=+2680.054237162"
Oct 30 21:47:03.133575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.133541    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:03.464809 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.464775    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:47:03.480935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.480897    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:47:03.976457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:03.969881    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:04.003473 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:04.001746    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:04.075189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:04.075159    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:04.076310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:04.076286    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:47:04.400195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:04.400165    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:05.071192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.071157    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:47:05.071679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.071372    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" containerID="cri-o://c687d922dcbcf69a1c8b9039aa2d00523216060ce8b6b227555369a77935dac1" gracePeriod=30
Oct 30 21:47:05.154582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.154531    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:47:05.178602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.178564    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" podUID="c9cd5082-c364-4118-83d0-5484a594bd36" containerName="maas-api" containerID="cri-o://1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824" gracePeriod=30
Oct 30 21:47:05.239223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.239172    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:05.850815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.850791    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:47:05.909733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.909690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:05.932894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.932864    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\" (UniqueName: \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\") pod \"c9cd5082-c364-4118-83d0-5484a594bd36\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") "
Oct 30 21:47:05.934142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.933037    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\") pod \"c9cd5082-c364-4118-83d0-5484a594bd36\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") "
Oct 30 21:47:05.934142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.933201    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\") pod \"c9cd5082-c364-4118-83d0-5484a594bd36\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") "
Oct 30 21:47:05.934142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.933262    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") pod \"c9cd5082-c364-4118-83d0-5484a594bd36\" (UID: \"c9cd5082-c364-4118-83d0-5484a594bd36\") "
Oct 30 21:47:05.934142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.933423    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca" pod "c9cd5082-c364-4118-83d0-5484a594bd36" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:05.934321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.934210    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9cd5082-c364-4118-83d0-5484a594bd36-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:05.935772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.935745    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs" pod "c9cd5082-c364-4118-83d0-5484a594bd36" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:05.936039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.936021    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s" pod "c9cd5082-c364-4118-83d0-5484a594bd36" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36"). InnerVolumeSpecName "kube-api-access-s897s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:05.936256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:05.936235    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls" pod "c9cd5082-c364-4118-83d0-5484a594bd36" (UID: "c9cd5082-c364-4118-83d0-5484a594bd36"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:47:06.027667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.027637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:06.035501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.035481    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:06.035501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.035501    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9cd5082-c364-4118-83d0-5484a594bd36-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:06.035679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.035519    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s897s\" (UniqueName: \"kubernetes.io/projected/c9cd5082-c364-4118-83d0-5484a594bd36-kube-api-access-s897s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:06.164458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.159578    6738 generic.go:356] "Generic (PLEG): container finished" podID="c9cd5082-c364-4118-83d0-5484a594bd36" containerID="1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824" exitCode=0
Oct 30 21:47:06.164458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.160489    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" event={"ID":"c9cd5082-c364-4118-83d0-5484a594bd36","Type":"ContainerDied","Data":"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"}
Oct 30 21:47:06.164458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.160551    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7" event={"ID":"c9cd5082-c364-4118-83d0-5484a594bd36","Type":"ContainerDied","Data":"937500f9292c213c811799b11bbbc19d31a598553fcf0deaca67f91b9cff151f"}
Oct 30 21:47:06.164458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.160582    6738 scope.go:122] "RemoveContainer" containerID="1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"
Oct 30 21:47:06.164458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.160786    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"
Oct 30 21:47:06.271468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.271133    6738 scope.go:122] "RemoveContainer" containerID="1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"
Oct 30 21:47:06.280458 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:06.273668    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824\": container with ID starting with 1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824 not found: ID does not exist" containerID="1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"
Oct 30 21:47:06.280458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.273761    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824"} err="failed to get container status \"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824\": rpc error: code = NotFound desc = could not find container \"1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824\": container with ID starting with 1ac10f70834b934509303cdd60a2a1725c019d03a9967f5891de2fde81394824 not found: ID does not exist"
Oct 30 21:47:06.319457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.313423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:06.335054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.334995    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:47:06.371945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.371772    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-729802-67c4d8bbd4-x4gn7"]
Oct 30 21:47:06.552513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.551485    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:47:06.556511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.553205    6738 cpu_manager.go:414] "containerMap: removing container" podUID="c9cd5082-c364-4118-83d0-5484a594bd36" containerName="maas-api"
Oct 30 21:47:06.556511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.553232    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9cd5082-c364-4118-83d0-5484a594bd36" containerName="maas-api"
Oct 30 21:47:06.556511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.553491    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9cd5082-c364-4118-83d0-5484a594bd36" containerName="maas-api"
Oct 30 21:47:06.567456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.562254    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.597266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.594902    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:47:06.665386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.665116    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.665386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.665267    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.665386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.665332    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5fdm\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.665739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.665409    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.697243 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.695184    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:06.720458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.716503    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.720458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.716674    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.720458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.716726    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.720458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.716762    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.720458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.716804    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.827646    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.827789    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.827927    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.827986    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5fdm\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.828066    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.828100    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.828227    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.828272    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.828301    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.830508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.829387    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.831864    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.832120    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.836407    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.837416    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.846040    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.873830    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.874886    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.878503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.877901    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5fdm\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\") pod \"e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") " pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.883056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.882755    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:06.955699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:06.955659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:07.225134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.225101    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9cd5082-c364-4118-83d0-5484a594bd36" path="/var/lib/kubelet/pods/c9cd5082-c364-4118-83d0-5484a594bd36/volumes"
Oct 30 21:47:07.698643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.698611    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:47:07.701214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.701169    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:07.701342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.701258    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:07.701342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.701306    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:47:07.882847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:07.882810    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.093601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.085874    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.309743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.309708    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" event={"ID":"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee","Type":"ContainerStarted","Data":"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"}
Oct 30 21:47:08.311290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.311257    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" event={"ID":"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee","Type":"ContainerStarted","Data":"fc04e3e8d8b5d4cd564b16c4ca800e90cfbfe4b94662d75a3f1304a9d390af44"}
Oct 30 21:47:08.359184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.359052    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podStartSLOduration=2.35902914 podStartE2EDuration="2.35902914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:08.353056556 +0000 UTC m=+2685.308743785" watchObservedRunningTime="2026-10-30 21:47:08.35902914 +0000 UTC m=+2685.314716370"
Oct 30 21:47:08.460141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.460107    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:08.471470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.469543    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing" containerID="cri-o://d746c9f2542ca18493f71bce14c6f88da92a543074e1fd27d3e850d4b35e51ef" gracePeriod=25
Oct 30 21:47:08.486477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.484035    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing" containerID="cri-o://cfc3d8882f55ee066809be5d659b65bf8ff96bd64bdd3048068136456785f957" gracePeriod=25
Oct 30 21:47:08.888466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.886998    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:08.910462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.902958    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:08.910462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:08.903014    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:09.913990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:09.913945    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:09.975461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:09.950116    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:10.169524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.163258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.293080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.174109    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:47:10.354461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.348821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:10.384088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.384043    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:47:10.387005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.384370    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.420595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.420524    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 21:47:10.420826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.420793    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 21:47:10.430501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.430464    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.430635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.430583    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.430700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.430669    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg6gx\" (UniqueName: \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.430949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.430930    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.431293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.431267    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-cf51d4-serving-cert\""
Oct 30 21:47:10.465464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.447908    6738 generic.go:356] "Generic (PLEG): container finished" podID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerID="c687d922dcbcf69a1c8b9039aa2d00523216060ce8b6b227555369a77935dac1" exitCode=0
Oct 30 21:47:10.465464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.448286    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" event={"ID":"8808f347-188b-4cf9-8d64-7946fdd523f1","Type":"ContainerDied","Data":"c687d922dcbcf69a1c8b9039aa2d00523216060ce8b6b227555369a77935dac1"}
Oct 30 21:47:10.537462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.537196    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.537462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.537277    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.537462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.537393    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.543471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.538186    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.564511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.554860    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:10.564511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.556180    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg6gx\" (UniqueName: \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.564511 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:10.557032    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-serving-cert: secret "maas-api-e2e-inf-a-cf51d4-serving-cert" not found
Oct 30 21:47:10.564511 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:10.557101    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls podName:f3ba5b05-aeac-4956-8360-f688e2aa8d55 nodeName:}" failed. No retries permitted until 2026-10-30 21:47:11.057080476 +0000 UTC m=+2688.012767697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls") pod "maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55") : secret "maas-api-e2e-inf-a-cf51d4-serving-cert" not found
Oct 30 21:47:10.592458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.579848    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:10.592458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.591128    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.644204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.644164    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg6gx\" (UniqueName: \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:10.903299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.897885    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:10.903299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:10.897941    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:11.009214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.009171    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:11.081246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.073616    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:11.182930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.181301    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:11.182930 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:11.181541    6738 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-serving-cert: secret "maas-api-e2e-inf-a-cf51d4-serving-cert" not found
Oct 30 21:47:11.182930 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:11.181613    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls podName:f3ba5b05-aeac-4956-8360-f688e2aa8d55 nodeName:}" failed. No retries permitted until 2026-10-30 21:47:12.181592241 +0000 UTC m=+2689.137279470 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls") pod "maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55") : secret "maas-api-e2e-inf-a-cf51d4-serving-cert" not found
Oct 30 21:47:11.353877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.353295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:11.374833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.374800    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:47:11.387655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.387616    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.387965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.387941    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.388122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.388105    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.388374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.388354    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.388624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.388603    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.388760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.388745    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.388964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.388947    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.389114 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.389099    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.389340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.389323    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\") pod \"8808f347-188b-4cf9-8d64-7946fdd523f1\" (UID: \"8808f347-188b-4cf9-8d64-7946fdd523f1\") "
Oct 30 21:47:11.390511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.390484    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:11.397798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.397774    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:11.398020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.397886    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:11.398520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.398498    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:11.407543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.407518    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:11.444761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.444708    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:47:11.445806 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.445774    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:11.446293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.446266    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:11.450836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.450796    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s" pod "8808f347-188b-4cf9-8d64-7946fdd523f1" (UID: "8808f347-188b-4cf9-8d64-7946fdd523f1"). InnerVolumeSpecName "kube-api-access-74k2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:11.470134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.470101    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6" event={"ID":"8808f347-188b-4cf9-8d64-7946fdd523f1","Type":"ContainerDied","Data":"d8fc08abb83a92ec1d955c2d71fa389ba1e6fb601f113d333465e1975dcb6c66"}
Oct 30 21:47:11.470359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.470339    6738 scope.go:122] "RemoveContainer" containerID="c687d922dcbcf69a1c8b9039aa2d00523216060ce8b6b227555369a77935dac1"
Oct 30 21:47:11.470703 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.470683    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494327    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74k2s\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-kube-api-access-74k2s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494362    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494380    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494398    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494411    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494425    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494483    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8808f347-188b-4cf9-8d64-7946fdd523f1-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494497    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8808f347-188b-4cf9-8d64-7946fdd523f1-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.496464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.494512    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8808f347-188b-4cf9-8d64-7946fdd523f1-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:11.541967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.541919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:11.547084 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.547053    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:47:11.626015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.625340    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-xjbf6"]
Oct 30 21:47:11.865619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.865580    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:47:11.867070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.867046    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:47:11.884108 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.884075    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:11.884230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:11.884118    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:12.292511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:12.279063    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:12.302471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:12.300781    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:12.548601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:12.548502    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:12.970465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:12.968475    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:12.970465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:12.968538    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:13.071677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.071605    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:13.248942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.246945    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" path="/var/lib/kubelet/pods/8808f347-188b-4cf9-8d64-7946fdd523f1/volumes"
Oct 30 21:47:13.264175 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:47:13.263426    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ba5b05_aeac_4956_8360_f688e2aa8d55.slice/crio-199985a08c41bc472a31a2931b0afe96d396aaf59652222489c91c3529365acb WatchSource:0}: Error finding container 199985a08c41bc472a31a2931b0afe96d396aaf59652222489c91c3529365acb: Status 404 returned error can't find the container with id 199985a08c41bc472a31a2931b0afe96d396aaf59652222489c91c3529365acb
Oct 30 21:47:13.264175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.263936    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:47:13.521491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.516819    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" event={"ID":"f3ba5b05-aeac-4956-8360-f688e2aa8d55","Type":"ContainerStarted","Data":"199985a08c41bc472a31a2931b0afe96d396aaf59652222489c91c3529365acb"}
Oct 30 21:47:13.668798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.668766    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:13.818513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.817861    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:13.883516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.883488    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:13.883635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:13.883540    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:14.070292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.065998    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:47:14.070292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.066290    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy" containerID="cri-o://d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761" gracePeriod=30
Oct 30 21:47:14.127459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.123314    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:14.562278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.562244    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" event={"ID":"f3ba5b05-aeac-4956-8360-f688e2aa8d55","Type":"ContainerStarted","Data":"e39604a6993d8dca1faa58786d6823104bf067e4956c6b20f5d3fab172df66d2"}
Oct 30 21:47:14.563176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.563157    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:14.648103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.647563    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" podStartSLOduration=5.092819081 podStartE2EDuration="5.647543566s" totalImagesPullingTime="554.724485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:14.645862128 +0000 UTC m=+2691.601549359" watchObservedRunningTime="2026-10-30 21:47:14.647543566 +0000 UTC m=+2691.603230796"
Oct 30 21:47:14.881100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.881060    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:14.883617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.883585    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:14.883738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:14.883632    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:15.868921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:15.868889    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:15.884123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:15.884086    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused" start-of-body=
Oct 30 21:47:15.884314 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:15.884145    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.159:15021/healthz/ready\": dial tcp 10.134.0.159:15021: connect: connection refused"
Oct 30 21:47:16.625843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:16.625809    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:16.889355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:16.889280    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:16.889851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:16.889687    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:16.890507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:16.890487    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:47:16.988255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:16.988224    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:17.503020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:17.502987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:18.289768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:18.289734    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.098152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.098120    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:19.402478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.402452    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.496984    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497037    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497134    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497164    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497224    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497256    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.497599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497392    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.498103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.497570    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.498231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.498216    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\") pod \"8d437ef5-cade-4b2b-9631-29fdb0036a18\" (UID: \"8d437ef5-cade-4b2b-9631-29fdb0036a18\") "
Oct 30 21:47:19.499485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.499462    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:19.499933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.499899    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:19.500096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.500075    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:19.500326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.500276    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:19.502828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.500966    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:19.503531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.503508    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:19.503941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.503910    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:47:19.533315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.526581    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "kube-api-access-xlj9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:19.533315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.526672    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo" pod "8d437ef5-cade-4b2b-9631-29fdb0036a18" (UID: "8d437ef5-cade-4b2b-9631-29fdb0036a18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:47:19.601038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601004    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601030    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8d437ef5-cade-4b2b-9631-29fdb0036a18-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601040    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601050    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlj9p\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-kube-api-access-xlj9p\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601061    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601069    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601081    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601093    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.601281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.601102    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8d437ef5-cade-4b2b-9631-29fdb0036a18-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:19.619611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.619578    6738 generic.go:356] "Generic (PLEG): container finished" podID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerID="d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761" exitCode=0
Oct 30 21:47:19.619740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.619645    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"
Oct 30 21:47:19.619740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.619666    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" event={"ID":"8d437ef5-cade-4b2b-9631-29fdb0036a18","Type":"ContainerDied","Data":"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"}
Oct 30 21:47:19.619740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.619738    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx" event={"ID":"8d437ef5-cade-4b2b-9631-29fdb0036a18","Type":"ContainerDied","Data":"d406082de20844f37b91be2565a538e5113525111cbd954cae1874aa516adf97"}
Oct 30 21:47:19.619892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.619767    6738 scope.go:122] "RemoveContainer" containerID="d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"
Oct 30 21:47:19.638707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.638689    6738 scope.go:122] "RemoveContainer" containerID="d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"
Oct 30 21:47:19.639067 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:19.639039    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761\": container with ID starting with d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761 not found: ID does not exist" containerID="d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"
Oct 30 21:47:19.639144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.639078    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761"} err="failed to get container status \"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761\": rpc error: code = NotFound desc = could not find container \"d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761\": container with ID starting with d90b96800e53e0188c2e747faa80d138104bb432e0fed89845b8cbfe77637761 not found: ID does not exist"
Oct 30 21:47:19.651107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.651089    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:47:19.656652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.656631    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-729802-openshift-default-6d76999bc7-p69vx"]
Oct 30 21:47:19.823575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:19.823504    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:20.258515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:20.258427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:20.490813 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:20.490783    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:20.958824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:20.958789    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:21.168298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:21.168257    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" path="/var/lib/kubelet/pods/8d437ef5-cade-4b2b-9631-29fdb0036a18/volumes"
Oct 30 21:47:21.584100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:21.584071    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:47:21.660388 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:21.660358    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:21.865372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:21.865337    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:47:21.865563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:21.865343    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:47:22.542815 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:22.542772    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:47:22.759224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:22.759195    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:23.469646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:23.469603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:24.369015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:24.368985    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:24.955334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:24.955297    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:25.065924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:25.065895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:25.217303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:25.217218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:26.125040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:26.125006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:26.826027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:26.825990    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:27.646490 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:27.642799    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:28.366140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.366091    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:28.708250 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.708218    6738 generic.go:356] "Generic (PLEG): container finished" podID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerID="cfc3d8882f55ee066809be5d659b65bf8ff96bd64bdd3048068136456785f957" exitCode=0
Oct 30 21:47:28.708653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.708503    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" event={"ID":"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a","Type":"ContainerDied","Data":"cfc3d8882f55ee066809be5d659b65bf8ff96bd64bdd3048068136456785f957"}
Oct 30 21:47:28.710525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.710448    6738 generic.go:356] "Generic (PLEG): container finished" podID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerID="d746c9f2542ca18493f71bce14c6f88da92a543074e1fd27d3e850d4b35e51ef" exitCode=0
Oct 30 21:47:28.710525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.710483    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" event={"ID":"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8","Type":"ContainerDied","Data":"d746c9f2542ca18493f71bce14c6f88da92a543074e1fd27d3e850d4b35e51ef"}
Oct 30 21:47:28.778832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.778806    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:47:28.815043 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.815015    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:47:28.822056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.822020    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\" (UniqueName: \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\") pod \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") "
Oct 30 21:47:28.822333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.822314    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\") pod \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\" (UID: \"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8\") "
Oct 30 21:47:28.822752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.822726    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume" pod "2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" (UID: "2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:28.823129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.823103    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:28.825244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.825216    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj" pod "2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" (UID: "2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8"). InnerVolumeSpecName "kube-api-access-9ghzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:28.924831 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.924796    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\" (UniqueName: \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\") pod \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") "
Oct 30 21:47:28.925014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.924851    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\") pod \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\" (UID: \"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a\") "
Oct 30 21:47:28.925316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.925288    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume" pod "8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" (UID: "8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:47:28.925693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.925675    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ghzj\" (UniqueName: \"kubernetes.io/projected/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8-kube-api-access-9ghzj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:28.925770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.925699    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:28.927217 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:28.927198    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p" pod "8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" (UID: "8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a"). InnerVolumeSpecName "kube-api-access-8k45p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:47:29.027618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.027541    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k45p\" (UniqueName: \"kubernetes.io/projected/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a-kube-api-access-8k45p\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:47:29.135378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.135345    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:29.722575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.722548    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"
Oct 30 21:47:29.722575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.722556    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf" event={"ID":"2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8","Type":"ContainerDied","Data":"e5afe2cfbd63e971d4d7c21cf4fb7ff66e82f85073e4de1ccd4d985054307873"}
Oct 30 21:47:29.723060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.722622    6738 scope.go:122] "RemoveContainer" containerID="d746c9f2542ca18493f71bce14c6f88da92a543074e1fd27d3e850d4b35e51ef"
Oct 30 21:47:29.724686 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.724665    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"
Oct 30 21:47:29.725187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.724722    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf" event={"ID":"8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a","Type":"ContainerDied","Data":"830ff7225db7fa18d9689243f23a91c562926f6bab695b9d8eeff84502b8abd9"}
Oct 30 21:47:29.736680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.736651    6738 scope.go:122] "RemoveContainer" containerID="cfc3d8882f55ee066809be5d659b65bf8ff96bd64bdd3048068136456785f957"
Oct 30 21:47:29.756322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.756296    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:47:29.762218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.762190    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-729802-74b98594cf-tptvf"]
Oct 30 21:47:29.773717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.773699    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:47:29.778908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.778889    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-729802-79648745dc-hrjdf"]
Oct 30 21:47:29.946486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:29.946452    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:30.644673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:30.644643    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.170522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.170489    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" path="/var/lib/kubelet/pods/2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8/volumes"
Oct 30 21:47:31.171195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.171169    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" path="/var/lib/kubelet/pods/8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a/volumes"
Oct 30 21:47:31.344012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.343980    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.537208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.537120    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:47:31.538303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538274    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing"
Oct 30 21:47:31.538303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538300    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538344    6738 cpu_manager.go:414] "containerMap: removing container" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538354    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538388    6738 cpu_manager.go:414] "containerMap: removing container" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538395    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538453    6738 cpu_manager.go:414] "containerMap: removing container" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy"
Oct 30 21:47:31.538589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538461    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy"
Oct 30 21:47:31.538919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538617    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="8808f347-188b-4cf9-8d64-7946fdd523f1" containerName="istio-proxy"
Oct 30 21:47:31.538919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538630    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ec13458-bcd8-4ff5-881b-5e4fdbf9bcb8" containerName="payload-processing"
Oct 30 21:47:31.538919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538645    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d14f0bd-4ac5-4a83-bb88-68d6d5828f2a" containerName="payload-pre-processing"
Oct 30 21:47:31.538919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.538657    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="8d437ef5-cade-4b2b-9631-29fdb0036a18" containerName="istio-proxy"
Oct 30 21:47:31.591051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.569017    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.591051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.577653    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-570f01\"/\"openshift-service-ca.crt\""
Oct 30 21:47:31.591051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.577793    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-570f01\"/\"test-model-570f01-kserve-self-signed-certs\""
Oct 30 21:47:31.591051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.577895    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-570f01\"/\"default-dockercfg-btc4h\""
Oct 30 21:47:31.591051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.577980    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-570f01\"/\"kube-root-ca.crt\""
Oct 30 21:47:31.593622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.592095    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:47:31.615542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.615511    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:31.681737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.681707    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.681909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.681742    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxft8\" (UniqueName: \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.681909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.681832    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.682029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.681914    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.682029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.682020    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.682151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.682109    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.783854    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.783997    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.784129    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.784291    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.784323    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxft8\" (UniqueName: \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.784690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.784400    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.817026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.792352    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.817026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.792654    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.817026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.792892    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.841460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.841375    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxft8\" (UniqueName: \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.875255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.875190    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.875834 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.875752    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\") pod \"test-model-570f01-kserve-7994b8dbb9-kllzg\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") " pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:31.909319 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:31.896690    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:32.289664 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:47:32.289630    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98068c9f_8e54_414b_aa66_73aae297f5c1.slice/crio-9e41e99040598492c1229ffadc0c0208be28956d66586c70cd8ceea17a884dda WatchSource:0}: Error finding container 9e41e99040598492c1229ffadc0c0208be28956d66586c70cd8ceea17a884dda: Status 404 returned error can't find the container with id 9e41e99040598492c1229ffadc0c0208be28956d66586c70cd8ceea17a884dda
Oct 30 21:47:32.290354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:32.290326    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:47:32.450059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:32.450021    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:32.829582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:32.829546    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" event={"ID":"98068c9f-8e54-414b-aa66-73aae297f5c1","Type":"ContainerStarted","Data":"16fa6b2e48fdaf72141c4a7387a47a0557daf48521d9aade76c3f180d5ab8150"}
Oct 30 21:47:32.829751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:32.829596    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" event={"ID":"98068c9f-8e54-414b-aa66-73aae297f5c1","Type":"ContainerStarted","Data":"9e41e99040598492c1229ffadc0c0208be28956d66586c70cd8ceea17a884dda"}
Oct 30 21:47:32.905123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:32.905089    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:33.118374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.118313    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" podStartSLOduration=2.118294227 podStartE2EDuration="2.118294227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:32.848623354 +0000 UTC m=+2709.804310585" watchObservedRunningTime="2026-10-30 21:47:33.118294227 +0000 UTC m=+2710.073981456"
Oct 30 21:47:33.120448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.120416    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:47:33.132580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.127957    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.144614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.143540    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-3d9d7b-serving-cert\""
Oct 30 21:47:33.149797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.148733    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:47:33.201056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.201020    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:33.227972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.227946    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.228170 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.228150    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gp5s\" (UniqueName: \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.228341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.228326    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.228402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.228390    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.330361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.330328    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.330919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.330413    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.330919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.330587    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.330919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.330705    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gp5s\" (UniqueName: \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.331142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.331120    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.333115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.333094    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.333328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.333309    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.333542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.333521    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:33.338981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.338960    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gp5s\" (UniqueName: \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\") pod \"maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.465090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.464998    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:33.943860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:33.924610    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:47:34.357532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:34.350223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:34.976867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:34.976829    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" event={"ID":"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a","Type":"ContainerStarted","Data":"4c5254f6dcb50fe9782eedbdd019c836afcc5b5ffeaa1c7dd08caf36b6d09621"}
Oct 30 21:47:34.977088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:34.976910    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" event={"ID":"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a","Type":"ContainerStarted","Data":"1dde1a543143e0281036a1cc1d460557d85f9b895fb9b2cf329ca87c801a26fb"}
Oct 30 21:47:34.999766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:34.999716    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" podStartSLOduration=1.460920369 podStartE2EDuration="1.999701649s" totalImagesPullingTime="538.78128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:34.994050593 +0000 UTC m=+2711.949737823" watchObservedRunningTime="2026-10-30 21:47:34.999701649 +0000 UTC m=+2711.955388916"
Oct 30 21:47:35.148146 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:35.148103    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:35.938494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:35.938419    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:36.044271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:36.044237    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:36.646911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:36.646869    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:37.514345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:37.514311    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:38.426601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:38.426563    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:39.224357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:39.224322    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:40.031576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:40.031540    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:40.725214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:40.725180    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:41.512462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:41.512055    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:41.905292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:41.905259    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:41.924478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:41.924450    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:42.057504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:42.057476    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:47:42.099522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:42.099492    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:42.126111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:42.126081    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:47:42.484788 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:42.484759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:42.608748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:42.608688    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:43.010867 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:43.010814    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:43.809463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:43.804414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:44.771081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:44.771049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:45.846427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:45.846394    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:46.707783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:46.707750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:46.872375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:46.872339    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.005769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:47.005690    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.207092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:47.206943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:47.676198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:47.676170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.109150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.109049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.125377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.125351    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:47:48.130634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.130609    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.137974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.137945    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-87ae68\""
Oct 30 21:47:48.147322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.147286    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:47:48.211491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.211458    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.211947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.211924    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-784xm\" (UniqueName: \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.280402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.280370    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.313564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.313526    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-784xm\" (UniqueName: \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.313815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.313797    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.314751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.314724    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.325912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.325881    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-784xm\" (UniqueName: \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\") pod \"payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") " pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.423369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.423315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:48.463762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.463725    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:48.976471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:48.976398    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:47:49.234647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:49.234610    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" event={"ID":"724f2199-282c-48c3-8e22-0075d8543f7a","Type":"ContainerStarted","Data":"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"}
Oct 30 21:47:49.235143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:49.234675    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" event={"ID":"724f2199-282c-48c3-8e22-0075d8543f7a","Type":"ContainerStarted","Data":"ecc208b319cccec0ca10a212b091dd86f7bb692be2c6bcec0170fdad14b6b4e5"}
Oct 30 21:47:49.235143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:49.234707    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:47:49.242691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:49.242664    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:49.280359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:49.280196    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" podStartSLOduration=1.280165105 podStartE2EDuration="1.280165105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:47:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:47:49.274169347 +0000 UTC m=+2726.229856577" watchObservedRunningTime="2026-10-30 21:47:49.280165105 +0000 UTC m=+2726.235852383"
Oct 30 21:47:50.355190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:50.355158    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:51.127737 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:51.127671    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:52.043617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:52.043582    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:52.511212 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:47:52.511170    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:47:53.166694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:53.166661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:54.621846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:54.621815    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:54.879088 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:54.878993    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.227912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:55.227833    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.487061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:55.486977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:55.806213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:55.805570    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.64:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:56.539494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:56.527068    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:56.545923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:56.545616    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:56.588463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:56.573875    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:56.749363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:56.749322    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:47:56.868195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:56.868071    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:58.442443 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:58.442405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:47:59.552613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:47:59.552553    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:00.250562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:00.250534    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:48:00.359214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:00.359182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:01.144613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:01.144578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:01.955311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:01.955270    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:03.008425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:03.008379    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:04.017625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:04.017590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:04.952782 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:04.952717    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:05.117594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.117559    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:48:05.118109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.117612    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:48:05.631053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.631018    6738 generic.go:356] "Generic (PLEG): container finished" podID="817ed678-747a-494d-979e-10c0170ef7ca" containerID="e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5" exitCode=0
Oct 30 21:48:05.631206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.631084    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerDied","Data":"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"}
Oct 30 21:48:05.631566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.631547    6738 scope.go:122] "RemoveContainer" containerID="e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"
Oct 30 21:48:05.633633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.633598    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:05.633727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.633676    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:05.633727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.633713    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:05.873296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:05.873247    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.259110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.259004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.448281 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.448240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.745051 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.745017    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:06.748506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.748403    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerStarted","Data":"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"}
Oct 30 21:48:06.754329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.753910    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:48:06.759839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.759810    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.773739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.773699    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:48:06.774040 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.774019    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-cf51d4-openshift-default-dockercfg-jq2sn\""
Oct 30 21:48:06.863835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.863801    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.864097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.864079    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.864196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.864125    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.864262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.864228    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n758k\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.864326 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.864304    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.865107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.864938    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.865107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.865086    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.865297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.865151    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.865297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.865234    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.963733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.963704    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:48:06.963960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.963938    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:06.964015 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.964000    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:06.966897 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.966877    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967003    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967115    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967162    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967238    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967256    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967304 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967287    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967328    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967368    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967481    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n758k\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967550    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967663    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967768    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.967982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.967963    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.969710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.969687    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.970149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.970124    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.976775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.976752    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:06.976915 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:06.976901    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n758k\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\") pod \"e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") " pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:07.055513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.055031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:07.173789 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.164773    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:07.228928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.226248    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:07.571201 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:48:07.571153    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b8f042_e5c6_41c1_805e_63835541c2f3.slice/crio-26c9c67fc8de655955ab6bb2facc4c4e878269b81dfa09726ed473c9f1cd42fe WatchSource:0}: Error finding container 26c9c67fc8de655955ab6bb2facc4c4e878269b81dfa09726ed473c9f1cd42fe: Status 404 returned error can't find the container with id 26c9c67fc8de655955ab6bb2facc4c4e878269b81dfa09726ed473c9f1cd42fe
Oct 30 21:48:07.574506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.574479    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:48:07.582592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.575123    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:07.582592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.575215    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:07.582592 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.575261    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:07.663938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.663279    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:07.911462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.903092    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:07.919308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.919241    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" event={"ID":"b0b8f042-e5c6-41c1-805e-63835541c2f3","Type":"ContainerStarted","Data":"26c9c67fc8de655955ab6bb2facc4c4e878269b81dfa09726ed473c9f1cd42fe"}
Oct 30 21:48:07.981534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.966830    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:07.981534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:07.966887    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:08.592462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:08.592136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:08.947070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:08.946777    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" event={"ID":"b0b8f042-e5c6-41c1-805e-63835541c2f3","Type":"ContainerStarted","Data":"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"}
Oct 30 21:48:08.963876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:08.963707    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:08.963876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:08.963757    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:08.975231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:08.975171    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podStartSLOduration=2.9751552930000003 podStartE2EDuration="2.975155293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:08.96801632 +0000 UTC m=+2745.923703548" watchObservedRunningTime="2026-10-30 21:48:08.975155293 +0000 UTC m=+2745.930842522"
Oct 30 21:48:09.168746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.168713    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:09.168941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.168765    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:09.170840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.170812    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:09.810460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.810022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:09.973331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.973298    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:09.973676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:09.973643    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:10.167858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:10.167820    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:10.168061 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:10.167867    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:10.966488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:10.965487    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:10.966488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:10.965548    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:11.153467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.152806    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:11.167601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.167576    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:11.167843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.167820    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:11.467291 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.467262    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:11.651869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.651837    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:11.964810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.964770    6738 patch_prober.go:29] interesting pod/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 21:48:11.965010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:11.964843    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 21:48:12.167413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:12.167357    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:12.167935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:12.167909    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:12.625461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:12.620897    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:12.944085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:12.943955    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:13.068902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.068872    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:48:13.070386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.070353    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:48:13.071227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.071199    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:48:13.129468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.129418    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:13.169260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.168884    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:13.169260 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:13.168949    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:14.130461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:14.130079    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:14.167196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:14.167162    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:14.167359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:14.167210    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:15.188461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.185859    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused" start-of-body=
Oct 30 21:48:15.188461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.185932    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.164:15021/healthz/ready\": dial tcp 10.134.0.164:15021: connect: connection refused"
Oct 30 21:48:15.337265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.337227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:15.805078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.805036    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:48:15.809353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.809331    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.814571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.814551    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8431de-openshift-default-dockercfg-w6wgl\""
Oct 30 21:48:15.835989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.835966    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:48:15.945482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945449    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945577    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945609    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945638    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945774    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945809    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn49q\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945838    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.945991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.945974    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:15.946062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:15.946048    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.047879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.047850    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048013    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048060    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048079    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048108    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048222 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048189    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048226    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn49q\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048271    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048341    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048385    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048503    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048519    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.048972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.048949    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.049300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.049278    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.050713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.050692    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.050843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.050824    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.065887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.065823    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn49q\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.066045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.066028    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\") pod \"e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") " pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.124932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.124901    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:16.172675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.172647    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:16.172962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.172942    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:16.174702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.174680    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:48:16.368773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.368745    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:16.487367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.486077    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:48:16.490695 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:48:16.490655    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a1cf896_b64d_4adb_9da1_c0de35fa842c.slice/crio-4e77054ffb7523d714d69188037e7e213ddd067fc692eaaa389587216aae80e3 WatchSource:0}: Error finding container 4e77054ffb7523d714d69188037e7e213ddd067fc692eaaa389587216aae80e3: Status 404 returned error can't find the container with id 4e77054ffb7523d714d69188037e7e213ddd067fc692eaaa389587216aae80e3
Oct 30 21:48:16.494847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.494802    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:16.494930 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.494893    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:16.494975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:16.494936    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:48:17.167851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:17.167820    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" event={"ID":"2a1cf896-b64d-4adb-9da1-c0de35fa842c","Type":"ContainerStarted","Data":"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"}
Oct 30 21:48:17.168028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:17.167873    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" event={"ID":"2a1cf896-b64d-4adb-9da1-c0de35fa842c","Type":"ContainerStarted","Data":"4e77054ffb7523d714d69188037e7e213ddd067fc692eaaa389587216aae80e3"}
Oct 30 21:48:17.217458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:17.208775    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:17.224866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:17.224325    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podStartSLOduration=2.224301258 podStartE2EDuration="2.224301258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:48:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:48:17.19389775 +0000 UTC m=+2754.149584977" watchObservedRunningTime="2026-10-30 21:48:17.224301258 +0000 UTC m=+2754.179988487"
Oct 30 21:48:18.125848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.125807    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:18.127796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.127763    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:48:18.127938 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.127836    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:48:18.316462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.314871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:18.444271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.444173    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:18.623911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:18.623871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:19.126798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:19.126764    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:48:19.127379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:19.127346    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:48:19.499634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:19.493647    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.125352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:20.125319    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:48:20.125547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:20.125383    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:48:20.484733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:20.484604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:20.628655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:20.628621    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:21.126158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:21.126121    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:48:21.126368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:21.126183    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:48:21.290462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:21.281985    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:22.110478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:22.106675    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:22.130574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:22.130109    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused" start-of-body=
Oct 30 21:48:22.130574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:22.130172    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.165:15021/healthz/ready\": dial tcp 10.134.0.165:15021: connect: connection refused"
Oct 30 21:48:22.558154 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:48:22.558108    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:48:22.941971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:22.941900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:23.291239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.291156    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:23.293506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.293475    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:23.297762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.297739    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:48:23.661362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.661312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:23.829010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.828974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:23.959961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:23.959876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:24.417776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:24.417747    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:24.545356 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:24.545320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:25.403704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:25.403662    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:26.294134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:26.294102    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:27.147639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:27.147612    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:28.106277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:28.106225    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:28.850065 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:28.850031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:30.181749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:30.181717    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:31.070940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:31.070868    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:31.847134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:31.847094    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:31.998666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:31.998636    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:32.237005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:32.236957    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:33.056458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:33.056412    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:33.935147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:33.935114    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:34.777863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:34.777798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:35.863266 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:35.863240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:36.623461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:36.621590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:37.550488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:37.548290    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:38.480359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:38.480327    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:39.248251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:39.248218    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:39.374463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:39.374280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:40.179559 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:40.179528    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:41.498982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:41.498941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:42.466034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:42.466000    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:42.965504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:42.965478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:43.102500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:43.102422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:43.724900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:43.724863    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:44.460082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:44.460012    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:45.280562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:45.280509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:46.397257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:46.397214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:47.483455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:47.478479    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:48.283120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:48.283066    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:48.774947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:48.774874    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:49.529172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:49.529139    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:50.301499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:50.301460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:50.935422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:50.935387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:51.283514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:51.279686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:51.536783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:51.536678    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:51.657634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:51.657585    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:52.114798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:52.114764    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:52.520632 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:48:52.520545    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:48:52.791517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:52.788628    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:53.733311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:53.733281    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:54.726924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:54.726892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:55.608004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:55.607936    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:56.674694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:56.674657    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:57.652358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:57.652327    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:58.593526 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:58.593489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:59.329268 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:59.329235    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:59.495343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:59.495309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:48:59.817059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:48:59.816951    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:00.010471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:00.006660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:00.342988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:00.342952    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:00.630697 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:00.630666    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:01.667858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:01.667821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.528155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:02.528115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:02.655498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:02.655405    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:03.381005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:03.380973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:04.184644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:04.184590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.097416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:05.097382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.297202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:05.297168    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:49:05.321423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:05.321378    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:49:05.444708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:05.444663    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:05.893928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:05.893893    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:06.980824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:06.980781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:07.717375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:07.717340    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:07.872460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:07.863213    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:08.663685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:08.662924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:09.670172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:09.670130    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.094751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:10.094670    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.214400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:10.211353    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.243087 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.243060    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-98afef9e5c37: secret "agc-cred-98afef9e5c37" not found
Oct 30 21:49:10.243313 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.243301    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-b3a133238e6c: secret "agc-cred-b3a133238e6c" not found
Oct 30 21:49:10.243396 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.243388    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh: [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:10.243551 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.243537    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials podName:f5627795-0bb4-4ece-89a6-75bb7bd67da7 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:10.743517701 +0000 UTC m=+2807.699204907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7") : [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:10.756940 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.756907    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-98afef9e5c37: secret "agc-cred-98afef9e5c37" not found
Oct 30 21:49:10.756940 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.756938    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-b3a133238e6c: secret "agc-cred-b3a133238e6c" not found
Oct 30 21:49:10.756940 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.756951    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh: [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:10.757574 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:10.757036    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials podName:f5627795-0bb4-4ece-89a6-75bb7bd67da7 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:11.757016492 +0000 UTC m=+2808.712703714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7") : [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:10.824191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:10.824157    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:10.956210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:10.956181    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.277996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:11.277960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.383460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:11.324187    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:49:11.693463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:11.677962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:11.836541 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:11.836510    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-98afef9e5c37: secret "agc-cred-98afef9e5c37" not found
Oct 30 21:49:11.836541 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:11.836537    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-b3a133238e6c: secret "agc-cred-b3a133238e6c" not found
Oct 30 21:49:11.836541 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:11.836547    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh: [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:11.836990 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:11.836604    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials podName:f5627795-0bb4-4ece-89a6-75bb7bd67da7 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:13.836589091 +0000 UTC m=+2810.792276297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7") : [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:12.054746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:12.054671    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:49:12.132576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:12.132543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.262289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:12.262255    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:12.977989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:12.977961    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:13.240276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:13.240192    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:13.896424 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:13.896385    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-98afef9e5c37: secret "agc-cred-98afef9e5c37" not found
Oct 30 21:49:13.896424 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:13.896418    6738 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-fb2b58/agc-cred-b3a133238e6c: secret "agc-cred-b3a133238e6c" not found
Oct 30 21:49:13.896424 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:13.896427    6738 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh: [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:13.896707 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:13.896507    6738 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials podName:f5627795-0bb4-4ece-89a6-75bb7bd67da7 nodeName:}" failed. No retries permitted until 2026-10-30 21:49:17.896493694 +0000 UTC m=+2814.852180901 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7") : [secret "agc-cred-98afef9e5c37" not found, secret "agc-cred-b3a133238e6c" not found]
Oct 30 21:49:14.105352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:14.105300    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:14.961602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:14.961574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:15.171212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:15.171172    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:15.939381 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:15.939348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.384332 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:16.384295    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" podUID="9a633f34-b01b-464a-8747-22120dab9b9b" containerName="payload-processing" containerID="cri-o://d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77" gracePeriod=25
Oct 30 21:49:16.719213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:16.719184    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:49:16.767294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:16.767264    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:16.895462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:16.895134    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\") pod \"9a633f34-b01b-464a-8747-22120dab9b9b\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") "
Oct 30 21:49:16.895462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:16.895345    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\") pod \"9a633f34-b01b-464a-8747-22120dab9b9b\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") "
Oct 30 21:49:17.086844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.036569    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\") pod \"9a633f34-b01b-464a-8747-22120dab9b9b\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") "
Oct 30 21:49:17.086844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.036780    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\") pod \"9a633f34-b01b-464a-8747-22120dab9b9b\" (UID: \"9a633f34-b01b-464a-8747-22120dab9b9b\") "
Oct 30 21:49:17.086844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.065200    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" podUID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" containerName="payload-processing" containerID="cri-o://a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba" gracePeriod=25
Oct 30 21:49:17.086844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.077859    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume" pod "9a633f34-b01b-464a-8747-22120dab9b9b" (UID: "9a633f34-b01b-464a-8747-22120dab9b9b"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:17.094521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.087354    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume" pod "9a633f34-b01b-464a-8747-22120dab9b9b" (UID: "9a633f34-b01b-464a-8747-22120dab9b9b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:17.143057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.143034    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.143267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.143252    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a633f34-b01b-464a-8747-22120dab9b9b-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.144091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.143075    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd" pod "9a633f34-b01b-464a-8747-22120dab9b9b" (UID: "9a633f34-b01b-464a-8747-22120dab9b9b"). InnerVolumeSpecName "kube-api-access-6bthd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:17.150507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.149020    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials" pod "9a633f34-b01b-464a-8747-22120dab9b9b" (UID: "9a633f34-b01b-464a-8747-22120dab9b9b"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:17.153471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.153428    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.199663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.187731    6738 generic.go:356] "Generic (PLEG): container finished" podID="9a633f34-b01b-464a-8747-22120dab9b9b" containerID="d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77" exitCode=0
Oct 30 21:49:17.199663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.187793    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"
Oct 30 21:49:17.199663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.187949    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" event={"ID":"9a633f34-b01b-464a-8747-22120dab9b9b","Type":"ContainerDied","Data":"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"}
Oct 30 21:49:17.199663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.187992    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl" event={"ID":"9a633f34-b01b-464a-8747-22120dab9b9b","Type":"ContainerDied","Data":"0a792bdf47662cba0c995ed0acab10398d3930aec82209c73f4919e818de08e8"}
Oct 30 21:49:17.199663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.188018    6738 scope.go:122] "RemoveContainer" containerID="d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"
Oct 30 21:49:17.253741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.246598    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bthd\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-kube-api-access-6bthd\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.253741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.246640    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/9a633f34-b01b-464a-8747-22120dab9b9b-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.270021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.269986    6738 scope.go:122] "RemoveContainer" containerID="d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"
Oct 30 21:49:17.270350 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:17.270326    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77\": container with ID starting with d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77 not found: ID does not exist" containerID="d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"
Oct 30 21:49:17.270424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.270363    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77"} err="failed to get container status \"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77\": rpc error: code = NotFound desc = could not find container \"d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77\": container with ID starting with d6c59cc6c90a58fc432e2179b93907b8b7d6d98779b407ec2a8d0793249a3f77 not found: ID does not exist"
Oct 30 21:49:17.273558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.273529    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:49:17.278394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.278374    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-0ea1ea/payload-processing-external-model-e2e-extmod-b-0ea1ea-5bb72tjxl"]
Oct 30 21:49:17.314736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.314698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.459739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.459696    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.500123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.491425    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:49:17.654967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.654935    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\") pod \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") "
Oct 30 21:49:17.655156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.655017    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\") pod \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") "
Oct 30 21:49:17.655226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.655193    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\") pod \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") "
Oct 30 21:49:17.655331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.655317    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\") pod \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\" (UID: \"f5627795-0bb4-4ece-89a6-75bb7bd67da7\") "
Oct 30 21:49:17.655398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.655327    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume" pod "f5627795-0bb4-4ece-89a6-75bb7bd67da7" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:17.655608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.655585    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume" pod "f5627795-0bb4-4ece-89a6-75bb7bd67da7" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:49:17.656157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.656136    6738 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-routing-overlay-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.656253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.656162    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f5627795-0bb4-4ece-89a6-75bb7bd67da7-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.657586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.657566    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj" pod "f5627795-0bb4-4ece-89a6-75bb7bd67da7" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7"). InnerVolumeSpecName "kube-api-access-2wlzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:17.657676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.657596    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials" pod "f5627795-0bb4-4ece-89a6-75bb7bd67da7" (UID: "f5627795-0bb4-4ece-89a6-75bb7bd67da7"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:17.758639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.758545    6738 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-provider-credentials\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.758639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.758569    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wlzj\" (UniqueName: \"kubernetes.io/projected/f5627795-0bb4-4ece-89a6-75bb7bd67da7-kube-api-access-2wlzj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:17.758639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.758609    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:17.883464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:17.882820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:18.052342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.052258    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:49:18.053275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053253    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9a633f34-b01b-464a-8747-22120dab9b9b" containerName="payload-processing"
Oct 30 21:49:18.053401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053280    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a633f34-b01b-464a-8747-22120dab9b9b" containerName="payload-processing"
Oct 30 21:49:18.053401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053308    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" containerName="payload-processing"
Oct 30 21:49:18.053401 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053314    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" containerName="payload-processing"
Oct 30 21:49:18.053518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053474    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a633f34-b01b-464a-8747-22120dab9b9b" containerName="payload-processing"
Oct 30 21:49:18.053518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.053488    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" containerName="payload-processing"
Oct 30 21:49:18.057617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.057599    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.074991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.074951    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-adf308-openshift-default-dockercfg-58ljd\""
Oct 30 21:49:18.080284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.080256    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:49:18.151997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.151964    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:18.165733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.165707    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.165877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.165782    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.165877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.165832    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.165965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.165950    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.166008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.165975    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.166079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.166063    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.166186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.166174    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.166238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.166206    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.166297 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.166237    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jb4\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.199860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.199827    6738 generic.go:356] "Generic (PLEG): container finished" podID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" containerID="a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba" exitCode=0
Oct 30 21:49:18.200035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.199871    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"
Oct 30 21:49:18.200035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.199966    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" event={"ID":"f5627795-0bb4-4ece-89a6-75bb7bd67da7","Type":"ContainerDied","Data":"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"}
Oct 30 21:49:18.200035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.200015    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh" event={"ID":"f5627795-0bb4-4ece-89a6-75bb7bd67da7","Type":"ContainerDied","Data":"730a64eff68a5dbb2357857db631ad4e180af33e7b9a9b8fa0012d3c8305304b"}
Oct 30 21:49:18.200208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.200046    6738 scope.go:122] "RemoveContainer" containerID="a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.271931    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jb4\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.272109    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.272238    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.272280    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.273100    6738 scope.go:122] "RemoveContainer" containerID="a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:18.273651    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba\": container with ID starting with a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba not found: ID does not exist" containerID="a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"
Oct 30 21:49:18.277486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.273689    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba"} err="failed to get container status \"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba\": rpc error: code = NotFound desc = could not find container \"a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba\": container with ID starting with a118810116e3719955c2740af102ee4df0956b1a1a36ee4b4e198f9130e1dbba not found: ID does not exist"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.364115    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.364168    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.364233    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.364423    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.364574    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.367522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.366173    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.394459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.391528    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:49:18.394459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.392939    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.394459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.393903    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.407461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.394721    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.407461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.401036    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.407461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.401274    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.463466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.463080    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-fb2b58/payload-processing-external-model-e2e-extmod-a-fb2b58-df86j22mh"]
Oct 30 21:49:18.495611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.490077    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.495611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.493288    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jb4\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.495611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.493898    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:18.541516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.521840    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\") pod \"e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") " pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:18.678408 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:18.678329    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:19.024997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.024858    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:49:19.029002 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:49:19.028976    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76714185_024e_496e_b444_f9d187dedf65.slice/crio-5584d15615bb7fa2c8fe36712cd5d22315074c14506ecb269013ad3d3a311acd WatchSource:0}: Error finding container 5584d15615bb7fa2c8fe36712cd5d22315074c14506ecb269013ad3d3a311acd: Status 404 returned error can't find the container with id 5584d15615bb7fa2c8fe36712cd5d22315074c14506ecb269013ad3d3a311acd
Oct 30 21:49:19.033012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.032967    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:19.033125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.033058    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:19.033125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.033102    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:49:19.171584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.171551    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a633f34-b01b-464a-8747-22120dab9b9b" path="/var/lib/kubelet/pods/9a633f34-b01b-464a-8747-22120dab9b9b/volumes"
Oct 30 21:49:19.172340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.172303    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5627795-0bb4-4ece-89a6-75bb7bd67da7" path="/var/lib/kubelet/pods/f5627795-0bb4-4ece-89a6-75bb7bd67da7/volumes"
Oct 30 21:49:19.217130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.217093    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerStarted","Data":"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"}
Oct 30 21:49:19.217305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.217158    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerStarted","Data":"5584d15615bb7fa2c8fe36712cd5d22315074c14506ecb269013ad3d3a311acd"}
Oct 30 21:49:19.238667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.238187    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podStartSLOduration=1.238168927 podStartE2EDuration="1.238168927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:19.236926258 +0000 UTC m=+2816.192613509" watchObservedRunningTime="2026-10-30 21:49:19.238168927 +0000 UTC m=+2816.193856153"
Oct 30 21:49:19.246837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.246750    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:19.678999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:19.678971    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:20.281654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.281620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:20.342298 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.342257    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main" containerID="cri-o://a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79" gracePeriod=45
Oct 30 21:49:20.365371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.365331    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" containerID="cri-o://218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c" gracePeriod=45
Oct 30 21:49:20.564877 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.564577    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": dial tcp 10.134.0.48:8000: connect: connection refused"
Oct 30 21:49:20.687153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.687119    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:20.738942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.738918    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:49:20.742772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.742753    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:49:20.772152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.772127    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:20.834746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.834652    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.835025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.834993    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.835195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.835178    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.835982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.835951    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.836140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.836124    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.836378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.836361    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.836583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.836553    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\" (UniqueName: \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.837491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.837470    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\" (UniqueName: \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.841166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.841143    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.841357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.841327    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\" (UniqueName: \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.841600 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.841582    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\") pod \"251b4c4c-cc07-47da-82bf-555c8eed0861\" (UID: \"251b4c4c-cc07-47da-82bf-555c8eed0861\") "
Oct 30 21:49:20.842048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.842026    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\") pod \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\" (UID: \"a2b80643-e89e-4ade-9e58-f4bf4d86749c\") "
Oct 30 21:49:20.846966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.835621    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.847085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.835863    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.847085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.841967    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.847085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.845103    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.847085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.845386    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.847553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.847199    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.853829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.853786    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "kube-api-access-8nm98". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:20.855934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.855618    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:20.855934 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.855868    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "kube-api-access-gld2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:49:20.856931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.856906    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.859275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.859196    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm" pod "a2b80643-e89e-4ade-9e58-f4bf4d86749c" (UID: "a2b80643-e89e-4ade-9e58-f4bf4d86749c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:49:20.859545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.859520    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs" pod "251b4c4c-cc07-47da-82bf-555c8eed0861" (UID: "251b4c4c-cc07-47da-82bf-555c8eed0861"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:49:20.948224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948180    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gld2g\" (UniqueName: \"kubernetes.io/projected/a2b80643-e89e-4ade-9e58-f4bf4d86749c-kube-api-access-gld2g\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948209    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948226    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nm98\" (UniqueName: \"kubernetes.io/projected/251b4c4c-cc07-47da-82bf-555c8eed0861-kube-api-access-8nm98\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948236    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948245    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948252    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948259    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948267    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/251b4c4c-cc07-47da-82bf-555c8eed0861-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948275    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948283    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2b80643-e89e-4ade-9e58-f4bf4d86749c-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948294    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2b80643-e89e-4ade-9e58-f4bf4d86749c-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:20.948514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:20.948306    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/251b4c4c-cc07-47da-82bf-555c8eed0861-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:49:21.263873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.263837    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.346505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.346091    6738 generic.go:356] "Generic (PLEG): container finished" podID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerID="a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79" exitCode=0
Oct 30 21:49:21.346505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.346362    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" event={"ID":"a2b80643-e89e-4ade-9e58-f4bf4d86749c","Type":"ContainerDied","Data":"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"}
Oct 30 21:49:21.346505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.346415    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv" event={"ID":"a2b80643-e89e-4ade-9e58-f4bf4d86749c","Type":"ContainerDied","Data":"c71d354cb87d73356b069056eacaff57b359ca962cfb514399315b94cd449b49"}
Oct 30 21:49:21.346505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.346464    6738 scope.go:122] "RemoveContainer" containerID="a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"
Oct 30 21:49:21.346949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.346645    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"
Oct 30 21:49:21.352223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.352189    6738 generic.go:356] "Generic (PLEG): container finished" podID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerID="218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c" exitCode=0
Oct 30 21:49:21.352664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.352644    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"
Oct 30 21:49:21.353308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.353026    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" event={"ID":"251b4c4c-cc07-47da-82bf-555c8eed0861","Type":"ContainerDied","Data":"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"}
Oct 30 21:49:21.353308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.353069    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x" event={"ID":"251b4c4c-cc07-47da-82bf-555c8eed0861","Type":"ContainerDied","Data":"15b922ee12443876ffc23ce240e94917efddd0fdc3a46a6ef82140f504d33399"}
Oct 30 21:49:21.353511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.353491    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:21.357396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.357176    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:49:21.372242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.372217    6738 scope.go:122] "RemoveContainer" containerID="a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"
Oct 30 21:49:21.372754 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:21.372620    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79\": container with ID starting with a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79 not found: ID does not exist" containerID="a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"
Oct 30 21:49:21.372754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.372657    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79"} err="failed to get container status \"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79\": rpc error: code = NotFound desc = could not find container \"a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79\": container with ID starting with a25ff5714116ac6f8edfcf496ac4d03a4049aeb8f9f67c1244626ea291103d79 not found: ID does not exist"
Oct 30 21:49:21.372754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.372673    6738 scope.go:122] "RemoveContainer" containerID="218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"
Oct 30 21:49:21.383059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.383039    6738 scope.go:122] "RemoveContainer" containerID="218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"
Oct 30 21:49:21.383509 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:21.383479    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c\": container with ID starting with 218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c not found: ID does not exist" containerID="218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"
Oct 30 21:49:21.383622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.383520    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c"} err="failed to get container status \"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c\": rpc error: code = NotFound desc = could not find container \"218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c\": container with ID starting with 218f70adba913e4649b0f728f16798eeb54ce8363e878324e19fe07f3cb08f5c not found: ID does not exist"
Oct 30 21:49:21.409345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.407201    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:49:21.413139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.413105    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-384164/distinct-2-w0-384164-kserve-7756874f96-jjprv"]
Oct 30 21:49:21.428689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.428625    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:49:21.442839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.442811    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-384164/premium-w0-384164-kserve-d9fbf8b88-xz45x"]
Oct 30 21:49:21.696612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.696501    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:21.979094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:21.979005    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:22.511074 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:22.511038    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:49:23.024961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:23.024922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:23.168495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:23.168426    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" path="/var/lib/kubelet/pods/251b4c4c-cc07-47da-82bf-555c8eed0861/volumes"
Oct 30 21:49:23.169071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:23.169055    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" path="/var/lib/kubelet/pods/a2b80643-e89e-4ade-9e58-f4bf4d86749c/volumes"
Oct 30 21:49:23.377347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:23.377318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:24.260964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:24.260916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:25.509462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:25.508998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:26.501616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:26.501025    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:27.606350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.606319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.917505    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924203    6738 cpu_manager.go:414] "containerMap: removing container" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main"
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924224    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main"
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924270    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main"
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924276    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main"
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924516    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="251b4c4c-cc07-47da-82bf-555c8eed0861" containerName="main"
Oct 30 21:49:27.927656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.924544    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2b80643-e89e-4ade-9e58-f4bf4d86749c" containerName="main"
Oct 30 21:49:27.938519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.938495    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:27.948565 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.948523    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:49:27.977464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.954405    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-001667\"/\"kube-root-ca.crt\""
Oct 30 21:49:27.977464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.954655    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-001667\"/\"openshift-service-ca.crt\""
Oct 30 21:49:27.977464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:27.958778    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-001667\"/\"default-dockercfg-stvkj\""
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.019307    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-001667\"/\"test-model-001667-kserve-self-signed-certs\""
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.024145    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.030560    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.030646    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.030714    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.030836    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.030980    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.041457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.031006    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt6pp\" (UniqueName: \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.140132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.140079    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.140424 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.140400    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.140650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.140630    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.140875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.140858    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.141120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.141097    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.141239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.141222    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt6pp\" (UniqueName: \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.145176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.145153    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.149901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.145602    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.150331 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.150307    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.153318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.153297    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.158735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.158711    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt6pp\" (UniqueName: \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.159874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.159853    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\") pod \"test-model-001667-kserve-659c99f5f5-8jjsp\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") " pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:28.269953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:28.269862    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:29.519106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:29.519063    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:49:29.906459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:29.904134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:30.611000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:30.610965    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" event={"ID":"49f7d339-a578-4f54-b05c-3a8f5709b17d","Type":"ContainerStarted","Data":"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"}
Oct 30 21:49:30.611421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:30.611016    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" event={"ID":"49f7d339-a578-4f54-b05c-3a8f5709b17d","Type":"ContainerStarted","Data":"d33ba0276655110b57fad47107960c2203bc1b492e6198a4ff49a3141a5e5383"}
Oct 30 21:49:30.635209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:30.635152    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" podStartSLOduration=3.635134033 podStartE2EDuration="3.635134033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:49:30.629363754 +0000 UTC m=+2827.585050981" watchObservedRunningTime="2026-10-30 21:49:30.635134033 +0000 UTC m=+2827.590821261"
Oct 30 21:49:31.050571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:31.049167    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:31.270755 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:31.270727    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:32.028007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:32.027972    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:33.046580 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:33.046544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:33.838041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:33.837903    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:34.790387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:34.790338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:35.558966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:35.558934    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:36.440874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:36.440842    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:37.732898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:37.732859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.271295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:38.271265    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:38.284510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:38.284482    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:38.447372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:38.447342    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:38.706001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:38.705969    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:49:39.286762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:39.286725    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.044370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:40.044341    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.268028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:40.267991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:40.769161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:40.769128    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:41.513037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:41.512998    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:41.648767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:41.648737    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:42.988013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:42.987979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:44.279141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:44.279102    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:44.641257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:44.641229    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:45.298461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:45.298304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:46.168364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:46.168325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:46.819227 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:46.819196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:47.044458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:47.037100    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:47.441920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:47.441888    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:47.774267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:47.774186    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:47.997220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:47.997176    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:48.562691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:48.562660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:49.820982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:49.820945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:50.582517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:50.582424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:50.720594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:50.720559    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.680407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:51.680377    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:51.874474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:51.874403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:52.516493 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:49:52.516452    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:49:52.778187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:52.778093    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:53.794458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:53.794226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:54.560516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:54.560482    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.121481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.121427    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.237695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.237661    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.390516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.390008    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:55.482647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.482620    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:49:55.492005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.491957    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" containerID="cri-o://3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898" gracePeriod=30
Oct 30 21:49:55.500080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.500051    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:49:55.500323 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.500295    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy" containerID="cri-o://3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5" gracePeriod=30
Oct 30 21:49:55.535468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:55.535415    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:56.292312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:56.292276    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.104965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:57.104929    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:57.862249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:57.862216    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:58.622457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:58.611479    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:59.542478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:59.542397    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:49:59.830347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:59.830265    6738 patch_prober.go:29] interesting pod/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:49:59.830347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:49:59.830335    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:50:00.246047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.246013    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:00.842321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.841935    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:50:00.871615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.871590    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:50:00.971233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.971200    6738 generic.go:356] "Generic (PLEG): container finished" podID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerID="3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898" exitCode=0
Oct 30 21:50:00.971423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.971251    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"
Oct 30 21:50:00.971423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.971290    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" event={"ID":"fc923af9-5b1f-4d22-8b7b-445d1d573027","Type":"ContainerDied","Data":"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"}
Oct 30 21:50:00.971423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.971355    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9" event={"ID":"fc923af9-5b1f-4d22-8b7b-445d1d573027","Type":"ContainerDied","Data":"4e09c66244f535d1477f4a08453fc74d8c51ae0f96dd2dbd6637c5d36a236a61"}
Oct 30 21:50:00.971423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.971389    6738 scope.go:122] "RemoveContainer" containerID="3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"
Oct 30 21:50:00.972965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.972904    6738 generic.go:356] "Generic (PLEG): container finished" podID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerID="3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5" exitCode=0
Oct 30 21:50:00.973135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.972983    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"
Oct 30 21:50:00.973135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.973020    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" event={"ID":"f99e4e11-1be4-4a3f-929d-ab29d50043f4","Type":"ContainerDied","Data":"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"}
Oct 30 21:50:00.973135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.973082    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f" event={"ID":"f99e4e11-1be4-4a3f-929d-ab29d50043f4","Type":"ContainerDied","Data":"7f4ce023972d36c699c7dc43b124049c52da94385090ec117b232a58a4ef6a19"}
Oct 30 21:50:00.983989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.983969    6738 scope.go:122] "RemoveContainer" containerID="3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"
Oct 30 21:50:00.984295 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:50:00.984275    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898\": container with ID starting with 3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898 not found: ID does not exist" containerID="3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"
Oct 30 21:50:00.984363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.984306    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898"} err="failed to get container status \"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898\": rpc error: code = NotFound desc = could not find container \"3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898\": container with ID starting with 3653d222e26f7936ef494794285d22e8e75306aa2a1af63e588bb3fcf3212898 not found: ID does not exist"
Oct 30 21:50:00.984363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.984323    6738 scope.go:122] "RemoveContainer" containerID="3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"
Oct 30 21:50:01.000077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.999399    6738 scope.go:122] "RemoveContainer" containerID="3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"
Oct 30 21:50:01.000077 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:50:00.999772    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5\": container with ID starting with 3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5 not found: ID does not exist" containerID="3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"
Oct 30 21:50:01.000077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:00.999800    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5"} err="failed to get container status \"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5\": rpc error: code = NotFound desc = could not find container \"3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5\": container with ID starting with 3396976a592c7f451010fd5f104fb2ec6f323eca9bf90f99b2d0aaf6612073c5 not found: ID does not exist"
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005543    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005604    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005647    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005677    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005755    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005806    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005858    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005888    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005935    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.005976    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006051    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006060    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006102    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006244    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006325    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006374    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006536    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006632    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006719    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\") pod \"fc923af9-5b1f-4d22-8b7b-445d1d573027\" (UID: \"fc923af9-5b1f-4d22-8b7b-445d1d573027\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006745    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006883    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\") pod \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\" (UID: \"f99e4e11-1be4-4a3f-929d-ab29d50043f4\") "
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.006977    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.007316    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008121    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008141    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008155    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008168    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc923af9-5b1f-4d22-8b7b-445d1d573027-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008182    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008358    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008536    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.008822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.008585    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.011550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.009955    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.011550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.010031    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:01.011550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.010149    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.014668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.014648    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "kube-api-access-xmnp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:01.014822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.014804    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "kube-api-access-kn7cs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:01.014913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.014888    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:50:01.014999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.014984    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.015135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.015113    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:50:01.015714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.015682    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token" pod "f99e4e11-1be4-4a3f-929d-ab29d50043f4" (UID: "f99e4e11-1be4-4a3f-929d-ab29d50043f4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:01.016980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.016955    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token" pod "fc923af9-5b1f-4d22-8b7b-445d1d573027" (UID: "fc923af9-5b1f-4d22-8b7b-445d1d573027"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:50:01.040985 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.040920    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130583    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmnp7\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-kube-api-access-xmnp7\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130619    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130635    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130649    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130663    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130680    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130696    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130714    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130729    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130744    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130760    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn7cs\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-kube-api-access-kn7cs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130772    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f99e4e11-1be4-4a3f-929d-ab29d50043f4-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.141498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.130784    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc923af9-5b1f-4d22-8b7b-445d1d573027-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:50:01.299249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.298775    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:50:01.303611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.303586    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-rcqc9"]
Oct 30 21:50:01.316231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.316209    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:50:01.320118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.320098    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-384164-openshift-default-c9776b589-hjx2f"]
Oct 30 21:50:01.894277 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:01.894246    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:02.591464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:02.586941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.174888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:03.174842    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" path="/var/lib/kubelet/pods/f99e4e11-1be4-4a3f-929d-ab29d50043f4/volumes"
Oct 30 21:50:03.175665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:03.175638    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" path="/var/lib/kubelet/pods/fc923af9-5b1f-4d22-8b7b-445d1d573027/volumes"
Oct 30 21:50:03.324244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:03.324208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:03.683970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:03.683939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.048111 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:04.048028    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:04.872460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:04.865111    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:05.718478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:05.715386    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:06.557564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:06.557519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:07.331073 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:07.331044    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:08.217050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:08.217008    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:09.468499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:09.468421    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:10.675341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:10.675304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:11.797464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:11.795392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:12.668611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:12.668584    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:13.461764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:13.461733    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:14.385798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:14.385721    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:15.167618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:15.167586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:16.227771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:16.227712    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:17.054721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:17.054686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:17.754244 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:17.754208    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:17.891093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:17.890841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.233857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:18.233786    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.647507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:18.647478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:18.794149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:18.794048    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:19.550841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:19.550805    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:20.552871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:20.552838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:20.718133 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:20.718089    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:21.218803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.218723    6738 generic.go:356] "Generic (PLEG): container finished" podID="76714185-024e-496e-b444-f9d187dedf65" containerID="662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf" exitCode=0
Oct 30 21:50:21.218994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.218795    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerDied","Data":"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"}
Oct 30 21:50:21.219267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.219250    6738 scope.go:122] "RemoveContainer" containerID="662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"
Oct 30 21:50:21.221589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.221550    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:21.221729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.221645    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:21.221729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.221693    6738 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 21:50:21.302601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.302539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:21.375919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:21.375817    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:50:22.234236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.234204    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerStarted","Data":"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"}
Oct 30 21:50:22.361914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.361880    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:22.553049 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:50:22.552965    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:50:22.679610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.679326    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:50:22.680247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.680218    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:22.680307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.680286    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:22.763300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.762346    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:22.908455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:22.905330    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:23.277207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:23.277124    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:23.679804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:23.679771    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:23.679989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:23.679818    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:23.996616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:23.987808    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.076309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.076265    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:50:24.077666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077640    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy"
Oct 30 21:50:24.077666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077667    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy"
Oct 30 21:50:24.077848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077739    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy"
Oct 30 21:50:24.077848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077746    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy"
Oct 30 21:50:24.077951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077933    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc923af9-5b1f-4d22-8b7b-445d1d573027" containerName="istio-proxy"
Oct 30 21:50:24.077998 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.077953    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f99e4e11-1be4-4a3f-929d-ab29d50043f4" containerName="istio-proxy"
Oct 30 21:50:24.082492 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.081776    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.143460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.122756    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:50:24.143460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.142745    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-e8d315\""
Oct 30 21:50:24.205558 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.197409    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.224670 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.199525    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-e8d315-dockercfg-p8tmz\""
Oct 30 21:50:24.283004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.278546    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dp2x\" (UniqueName: \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.283004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.278742    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.416466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.380623    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.416466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.380907    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dp2x\" (UniqueName: \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.416466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.382293    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.546542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.534919    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:24.585458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.578122    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dp2x\" (UniqueName: \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\") pod \"payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") " pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:24.681586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.681531    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:24.681586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.681582    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:24.832591 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:24.832489    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:25.138509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.138474    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:50:25.311011 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.310895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:25.450522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.449912    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" event={"ID":"e374033d-2102-4de2-8520-28eb00acdd15","Type":"ContainerStarted","Data":"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"}
Oct 30 21:50:25.450522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.449960    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" event={"ID":"e374033d-2102-4de2-8520-28eb00acdd15","Type":"ContainerStarted","Data":"f3472e15367d9e3735080fae246f7980846180941400f732cc57cbb875d46347"}
Oct 30 21:50:25.450522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.449995    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:25.470102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.470045    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" podStartSLOduration=1.470031465 podStartE2EDuration="1.470031465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:50:25.468190811 +0000 UTC m=+2882.423878038" watchObservedRunningTime="2026-10-30 21:50:25.470031465 +0000 UTC m=+2882.425718691"
Oct 30 21:50:25.685365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.684813    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:25.685365 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:25.684907    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:26.373048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:26.373015    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:26.679472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:26.679423    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:26.679653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:26.679490    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:27.281823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:27.281795    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:27.679264 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:27.679233    6738 patch_prober.go:29] interesting pod/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused" start-of-body=
Oct 30 21:50:27.679662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:27.679279    6738 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.166:15021/healthz/ready\": dial tcp 10.134.0.166:15021: connect: connection refused"
Oct 30 21:50:28.199747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:28.199697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:28.684009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:28.683976    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:50:28.684412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:28.684391    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:50:28.685078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:28.685058    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:50:29.088364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:29.088249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:29.851953 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:29.851922    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:30.537059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:30.537025    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:31.241117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:31.241080    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:32.027680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:32.027615    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:32.764508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:32.764470    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:33.557941 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:33.557906    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:34.360708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:34.360676    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.255496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:35.255459    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:35.941252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:35.941227    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:36.545504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:36.545479    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:50:36.749150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:36.749115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:37.443267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:37.443216    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.126842 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:38.126807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:38.852311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:38.852279    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:39.621881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:39.621852    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:40.337652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:40.337613    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:40.976689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:40.976660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:41.724354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:41.724318    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:42.433445 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:42.433413    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:43.218368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:43.218302    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:43.949486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:43.949449    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.712615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:44.712565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.841464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:44.834823    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:44.959799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:44.959762    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:45.598886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:45.598850    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:46.472426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:46.472389    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:47.336740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:47.336705    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:48.203858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:48.203824    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.037609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:49.037564    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:49.958955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:49.958036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:50.757685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:50.757637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:51.534037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:51.534004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:52.255080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:52.255045    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:52.512171 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:50:52.512089    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:50:53.047751 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:53.047702    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:53.726928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:53.726895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:54.596894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:54.596849    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:55.387067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:55.385949    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.170233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:56.170196    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:56.891878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:56.891832    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:57.647569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:57.647529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:58.346944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:58.346910    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:50:59.268164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:50:59.267612    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:00.003873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:00.003840    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:00.674609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:00.674580    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:01.352383 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:01.352352    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:02.074664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:02.074626    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:02.868038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:02.868006    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:03.638358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:03.638320    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:04.634827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:04.634791    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:05.234033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:05.234004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:06.036883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:06.036837    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:06.742799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:06.742748    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:07.420735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:07.420698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:08.478012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:08.477973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:09.214508 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:09.214478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:10.143389 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:10.143341    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:11.191973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:11.191943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:12.067706 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:12.067662    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:12.918168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:12.918136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:13.667533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:13.667501    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:13.787211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:13.787173    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:14.562545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:14.562513    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.328476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.328408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.815494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.811916    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:51:15.819940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.819916    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:51:15.820137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.820052    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.828675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.828649    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-adf308\"/\"openshift-service-ca.crt\""
Oct 30 21:51:15.835457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.829134    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-adf308\"/\"test-model-adf308-kserve-self-signed-certs\""
Oct 30 21:51:15.835624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.835274    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-adf308\"/\"default-dockercfg-zvl7k\""
Oct 30 21:51:15.835764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.835341    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-adf308\"/\"kube-root-ca.crt\""
Oct 30 21:51:15.869976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.869948    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:15.910257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910229    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.910413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910382    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.910556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910518    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.910657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910639    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.910725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910701    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:15.910790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:15.910773    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5l5c\" (UniqueName: \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.012087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.012016    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.012479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.012458    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.012645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.012630    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.012827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.012810    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5l5c\" (UniqueName: \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.013020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.013006    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.013310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.013292    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.014071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.014019    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.014613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.014591    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.014872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.014849    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.016772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.016296    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.023892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.023844    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5l5c\" (UniqueName: \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.027035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.027011    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\") pod \"test-model-adf308-kserve-8678df9559-m86nk\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") " pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.040426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.040361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:16.168757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.168520    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:16.606395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.606315    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:51:16.615888 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:51:16.615854    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b1a1130_125c_4344_8efc_00466623cf8b.slice/crio-922139d3607904ed9608b5958e1ee9ba37cd43da5dbe659efc32d1fadd18b6bc WatchSource:0}: Error finding container 922139d3607904ed9608b5958e1ee9ba37cd43da5dbe659efc32d1fadd18b6bc: Status 404 returned error can't find the container with id 922139d3607904ed9608b5958e1ee9ba37cd43da5dbe659efc32d1fadd18b6bc
Oct 30 21:51:16.844726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:16.844693    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:17.346033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:17.345993    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" event={"ID":"4b1a1130-125c-4344-8efc-00466623cf8b","Type":"ContainerStarted","Data":"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"}
Oct 30 21:51:17.346238 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:17.346044    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" event={"ID":"4b1a1130-125c-4344-8efc-00466623cf8b","Type":"ContainerStarted","Data":"922139d3607904ed9608b5958e1ee9ba37cd43da5dbe659efc32d1fadd18b6bc"}
Oct 30 21:51:17.364870 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:17.364810    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" podStartSLOduration=2.364797652 podStartE2EDuration="2.364797652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 21:51:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:51:17.363826574 +0000 UTC m=+2934.319513803" watchObservedRunningTime="2026-10-30 21:51:17.364797652 +0000 UTC m=+2934.320484879"
Oct 30 21:51:17.552963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:17.552930    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:18.169573 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:18.169544    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:18.218629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:18.218599    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:19.030645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:19.030607    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:19.814423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:19.814385    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:20.653546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:20.653513    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:20.942586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:20.942514    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:21.520157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:21.520112    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:22.241687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:22.241657    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:22.513707 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:51:22.513613    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:51:22.931486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:22.931449    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:23.760520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:23.760483    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:24.462158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:24.462119    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:25.334292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:25.334258    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:26.058972 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:26.058941    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:26.169168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:26.169140    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:26.184271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:26.184247    6738 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:26.452853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:26.452818    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:51:26.823718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:26.823621    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:27.650846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:27.650811    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:27.857167 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:27.857116    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:28.565902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:28.565821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:29.345562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:29.345534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:30.240523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:30.240477    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:30.941345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:30.941311    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:31.641071 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:31.641032    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:32.357313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:32.357280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:33.032189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:33.032151    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:33.720691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:33.720659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:34.454512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:34.454480    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:35.126808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:35.126773    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:35.771307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:35.771276    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:36.459284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:36.459257    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:37.220515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:37.220466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:38.009016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:38.008979    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:38.696455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:38.696403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:39.534727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:39.534660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:40.909997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:40.909962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:41.618691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:41.618651    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:42.336010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:42.335963    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:43.216933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:43.216905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:43.934003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:43.933973    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:44.633659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:44.633627    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:45.378651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:45.378601    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:46.072665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:46.072614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:46.822456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:46.822414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:47.532805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:47.532774    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:48.380083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:48.380043    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:48.804908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:48.804812    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:49.152428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:49.152392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:49.855053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:49.855026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:50.599746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:50.599691    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:51.338259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:51.338216    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:52.022608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:52.022572    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:52.512241 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:51:52.512203    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:51:52.754087 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:52.754055    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:53.521525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:53.521491    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:54.245717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:54.245683    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:55.023675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:55.023646    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:55.755074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:55.755042    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:56.430529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:56.430487    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:57.156986 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:57.156934    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.000605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:58.000529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:58.702909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:58.702872    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:51:59.510460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:51:59.504519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:00.540721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:00.540685    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:01.337676 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:01.337646    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:02.233042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:02.232989    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.200207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:03.200170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:03.959278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:03.959233    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:04.108255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:04.108210    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:52:04.108509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:04.108479    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy" containerID="cri-o://4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06" gracePeriod=30
Oct 30 21:52:04.742341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:04.742261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:05.566733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:05.566698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:06.322284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:06.322242    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:07.028932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:07.028894    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:52:07.029366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:07.029132    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy" containerID="cri-o://72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548" gracePeriod=30
Oct 30 21:52:07.198245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:07.198207    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:07.856736 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:07.856703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:08.533507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:08.533472    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:09.176830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.176804    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:09.446394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.446366    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:52:09.528511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.527864    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.528780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.528758    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.528970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.528952    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.529557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.529531    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.529770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.529752    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.530018 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.530000    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.530498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.530475    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.530760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.530739    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.538569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.534544    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:09.538569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.534899    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:09.538569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.535203    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:09.538569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.535259    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:09.538569 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.535585    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:09.548373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.548348    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.561346    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\") pod \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\" (UID: \"321c2bf4-2dc7-4980-adc5-6d3abc6a8588\") "
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562531    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562553    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562568    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562584    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562603    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.562661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.562617    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.564821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.564791    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:09.565987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.565963    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "kube-api-access-bqhsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:09.566089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.566011    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token" pod "321c2bf4-2dc7-4980-adc5-6d3abc6a8588" (UID: "321c2bf4-2dc7-4980-adc5-6d3abc6a8588"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:09.664033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.663934    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.664033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.663968    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqhsm\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-kube-api-access-bqhsm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.664033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.663982    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/321c2bf4-2dc7-4980-adc5-6d3abc6a8588-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:09.939466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:09.939332    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:10.038059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.038019    6738 generic.go:356] "Generic (PLEG): container finished" podID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerID="4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06" exitCode=0
Oct 30 21:52:10.038274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.038117    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"
Oct 30 21:52:10.038345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.038308    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" event={"ID":"321c2bf4-2dc7-4980-adc5-6d3abc6a8588","Type":"ContainerDied","Data":"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"}
Oct 30 21:52:10.038420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.038402    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x" event={"ID":"321c2bf4-2dc7-4980-adc5-6d3abc6a8588","Type":"ContainerDied","Data":"d1779c14915433b671b220b31dbe3525c5fcf6b8714e1f7c91cf3fb602580275"}
Oct 30 21:52:10.038535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.038488    6738 scope.go:122] "RemoveContainer" containerID="4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"
Oct 30 21:52:10.070974 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.068319    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:52:10.072665 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.072590    6738 scope.go:122] "RemoveContainer" containerID="4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"
Oct 30 21:52:10.072981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.072941    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-f1ed97-openshift-default-6bd85645bf-p4m7x"]
Oct 30 21:52:10.073064 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:52:10.073037    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06\": container with ID starting with 4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06 not found: ID does not exist" containerID="4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"
Oct 30 21:52:10.073124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.073078    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06"} err="failed to get container status \"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06\": rpc error: code = NotFound desc = could not find container \"4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06\": container with ID starting with 4ae49f20cc0911ec98e2582cdf56bf40923dba0814501b8363bb5d80cf435c06 not found: ID does not exist"
Oct 30 21:52:10.730322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:10.730287    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:11.168017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:11.167981    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" path="/var/lib/kubelet/pods/321c2bf4-2dc7-4980-adc5-6d3abc6a8588/volumes"
Oct 30 21:52:11.417131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:11.417053    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:12.171211 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.171182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:12.402784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.402755    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:52:12.434093 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434052    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434114    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434189    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434265    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434313    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434353    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434404    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434639    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.434717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.434702    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\") pod \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\" (UID: \"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee\") "
Oct 30 21:52:12.437496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.435926    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:12.437496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.436043    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:12.437496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.436414    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:12.437496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.437266    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:12.437496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.437455    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:12.441981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.441941    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:12.442461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.442419    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:12.442799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.442768    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "kube-api-access-s5fdm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:12.443229 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.443197    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy" pod "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" (UID: "3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:12.537035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.536995    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537035 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537033    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537052    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537070    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5fdm\" (UniqueName: \"kubernetes.io/projected/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-kube-api-access-s5fdm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537086    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537101    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537116    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537130    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.537233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.537143    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:12.943807 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:12.943777    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:13.075771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.075715    6738 generic.go:356] "Generic (PLEG): container finished" podID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerID="72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548" exitCode=0
Oct 30 21:52:13.075921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.075776    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"
Oct 30 21:52:13.075921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.075786    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" event={"ID":"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee","Type":"ContainerDied","Data":"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"}
Oct 30 21:52:13.075921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.075847    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd" event={"ID":"3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee","Type":"ContainerDied","Data":"fc04e3e8d8b5d4cd564b16c4ca800e90cfbfe4b94662d75a3f1304a9d390af44"}
Oct 30 21:52:13.075921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.075873    6738 scope.go:122] "RemoveContainer" containerID="72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"
Oct 30 21:52:13.088633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.088611    6738 scope.go:122] "RemoveContainer" containerID="72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"
Oct 30 21:52:13.088954 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:52:13.088935    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548\": container with ID starting with 72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548 not found: ID does not exist" containerID="72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"
Oct 30 21:52:13.089020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.088962    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548"} err="failed to get container status \"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548\": rpc error: code = NotFound desc = could not find container \"72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548\": container with ID starting with 72b05b6f3479f579ada9788d890220bbcae661ad39d0586edd5441685abf7548 not found: ID does not exist"
Oct 30 21:52:13.105581 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.102968    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:52:13.108991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.108968    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-2bvhd"]
Oct 30 21:52:13.168318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.168288    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" path="/var/lib/kubelet/pods/3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee/volumes"
Oct 30 21:52:13.737761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:13.737725    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:14.413141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:14.413098    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:15.231537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:15.231504    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:16.091463 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:16.090666    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:16.237480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:16.234539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:17.027305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:17.027272    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:17.935168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:17.935136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:18.736766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:18.736732    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:19.500704 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:19.500665    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:20.243566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:20.243534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:20.761914 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:20.761867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:21.119613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:21.117490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:21.857178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:21.857146    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:22.510547 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:52:22.510512    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:52:23.071077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:23.071028    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:24.025511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:24.025424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:24.937186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:24.937150    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:25.778278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:25.778197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:26.475937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:26.475900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:27.195101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:27.195069    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:27.225731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:27.225699    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:52:27.339557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:27.339509    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:28.519761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:28.519704    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:29.191805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:29.191772    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:29.953746 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:29.953706    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:30.809350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:30.809315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:31.467688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:31.467659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:31.570988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:31.570921    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:52:31.571203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:31.571183    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy" containerID="cri-o://1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288" gracePeriod=30
Oct 30 21:52:32.238201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:32.238168    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:32.351220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:32.351161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:33.004417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:33.004381    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:34.055833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:34.055802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:34.874795 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:34.874761    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:35.674359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:35.674317    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:36.437468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.434923    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:36.868814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.868786    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:52:36.932620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.932588    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.932802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.932788    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.933263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933232    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:36.933597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933553    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.933716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933644    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.933716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933675    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.933846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933723    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.933908 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933835    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:36.933965 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933910    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.934022 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.933966    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.934079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.934028    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:36.934079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.934065    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\") pod \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\" (UID: \"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03\") "
Oct 30 21:52:36.934968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.934945    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:36.935186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.935169    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:36.935287 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.935274    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:36.935514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.935490    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:36.935759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.935717    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:36.937684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.937640    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "kube-api-access-zwcvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:36.938295 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.938271    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:36.938550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.938527    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:36.938649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:36.938526    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo" pod "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" (UID: "a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:52:37.037364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037330    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.037364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037362    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwcvj\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-kube-api-access-zwcvj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.037364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037372    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.037648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037381    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.037648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037394    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.037648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.037405    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:37.360881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.360777    6738 generic.go:356] "Generic (PLEG): container finished" podID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerID="1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288" exitCode=0
Oct 30 21:52:37.360881 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.360834    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"
Oct 30 21:52:37.361112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.360878    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" event={"ID":"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03","Type":"ContainerDied","Data":"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"}
Oct 30 21:52:37.361112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.360934    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5" event={"ID":"a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03","Type":"ContainerDied","Data":"3a97d29420a0066c29a5213ec1d998ca238986c789aba4683b82271803ff9ed8"}
Oct 30 21:52:37.361112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.360958    6738 scope.go:122] "RemoveContainer" containerID="1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"
Oct 30 21:52:37.372650 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.372632    6738 scope.go:122] "RemoveContainer" containerID="1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"
Oct 30 21:52:37.372970 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:52:37.372952    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288\": container with ID starting with 1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288 not found: ID does not exist" containerID="1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"
Oct 30 21:52:37.373032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.372982    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288"} err="failed to get container status \"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288\": rpc error: code = NotFound desc = could not find container \"1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288\": container with ID starting with 1f80d76c0f8b344b2414dcb18d82e4b78d508d4a8497a36be8cda08688deb288 not found: ID does not exist"
Oct 30 21:52:37.429911 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.429867    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:52:37.465347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.465275    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:37.465708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.465681    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-4nzn5"]
Oct 30 21:52:37.874997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:37.874960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:38.016792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:38.016763    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:38.539162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:38.539111    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:39.169444 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:39.169396    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" path="/var/lib/kubelet/pods/a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03/volumes"
Oct 30 21:52:39.313246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:39.313039    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:40.055292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:40.055260    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:40.718063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:40.718003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:41.562464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:41.562417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:42.259648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.259587    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" podUID="98068c9f-8e54-414b-aa66-73aae297f5c1" containerName="main" containerID="cri-o://16fa6b2e48fdaf72141c4a7387a47a0557daf48521d9aade76c3f180d5ab8150" gracePeriod=45
Oct 30 21:52:42.336005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.335981    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:42.417572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.417525    6738 generic.go:356] "Generic (PLEG): container finished" podID="98068c9f-8e54-414b-aa66-73aae297f5c1" containerID="16fa6b2e48fdaf72141c4a7387a47a0557daf48521d9aade76c3f180d5ab8150" exitCode=0
Oct 30 21:52:42.417572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.417557    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" event={"ID":"98068c9f-8e54-414b-aa66-73aae297f5c1","Type":"ContainerDied","Data":"16fa6b2e48fdaf72141c4a7387a47a0557daf48521d9aade76c3f180d5ab8150"}
Oct 30 21:52:42.594852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.594823    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:52:42.735039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735009    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735082    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\" (UniqueName: \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735187    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735304    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:42.735418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735367    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735450    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:42.735551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735513    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\" (UniqueName: \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735551    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\") pod \"98068c9f-8e54-414b-aa66-73aae297f5c1\" (UID: \"98068c9f-8e54-414b-aa66-73aae297f5c1\") "
Oct 30 21:52:42.735613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.735602    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:42.736156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.736137    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.736253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.736159    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.736253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.736172    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.738123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.738097    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "kube-api-access-hxft8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:42.738278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.738256    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:42.738554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.738532    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm" pod "98068c9f-8e54-414b-aa66-73aae297f5c1" (UID: "98068c9f-8e54-414b-aa66-73aae297f5c1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:52:42.838404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.838331    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxft8\" (UniqueName: \"kubernetes.io/projected/98068c9f-8e54-414b-aa66-73aae297f5c1-kube-api-access-hxft8\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.838404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.838365    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/98068c9f-8e54-414b-aa66-73aae297f5c1-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.838404 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.838377    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98068c9f-8e54-414b-aa66-73aae297f5c1-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:42.975601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:42.975571    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:43.430315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.430232    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"
Oct 30 21:52:43.430315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.430239    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg" event={"ID":"98068c9f-8e54-414b-aa66-73aae297f5c1","Type":"ContainerDied","Data":"9e41e99040598492c1229ffadc0c0208be28956d66586c70cd8ceea17a884dda"}
Oct 30 21:52:43.430315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.430308    6738 scope.go:122] "RemoveContainer" containerID="16fa6b2e48fdaf72141c4a7387a47a0557daf48521d9aade76c3f180d5ab8150"
Oct 30 21:52:43.456471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.452699    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:52:43.472251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.472222    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-570f01/test-model-570f01-kserve-7994b8dbb9-kllzg"]
Oct 30 21:52:43.693607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:43.693524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:44.432709 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:44.432679    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:45.166453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:45.166402    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98068c9f-8e54-414b-aa66-73aae297f5c1" path="/var/lib/kubelet/pods/98068c9f-8e54-414b-aa66-73aae297f5c1/volumes"
Oct 30 21:52:45.242528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:45.242502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:46.057810 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:46.057781    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:46.944907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:46.944865    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:47.818060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:47.818026    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:48.460023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:48.459990    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:49.233048 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:49.233019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:49.894990 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:49.894958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:50.523649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:50.523620    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:51.278678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:51.278642    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:51.996089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:51.996056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:52.519080 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:52:52.519038    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:52:52.657150 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:52.657118    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:53.360357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:53.360321    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:54.023148 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:54.023115    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:54.822875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:54.822838    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:55.939683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:55.939600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:56.784003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:56.783970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:57.368012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.366002    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:52:57.368012 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.366235    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerName="maas-api" containerID="cri-o://4f69656f4f3b7fbe9a6f6392779043bd6044a068b0b2dd3cfd6dae3f6d39837e" gracePeriod=30
Oct 30 21:52:57.452069 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.452034    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:57.563123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.563083    6738 generic.go:356] "Generic (PLEG): container finished" podID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerID="4f69656f4f3b7fbe9a6f6392779043bd6044a068b0b2dd3cfd6dae3f6d39837e" exitCode=0
Oct 30 21:52:57.563296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.563159    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" event={"ID":"dc75253b-5d97-4da6-8fad-8fcd8245639a","Type":"ContainerDied","Data":"4f69656f4f3b7fbe9a6f6392779043bd6044a068b0b2dd3cfd6dae3f6d39837e"}
Oct 30 21:52:57.638545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.637451    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:57.801462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:57.792081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:58.045184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.045155    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:52:58.100655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.100622    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:58.122596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.122570    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\") pod \"dc75253b-5d97-4da6-8fad-8fcd8245639a\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") "
Oct 30 21:52:58.122778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.122636    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\") pod \"dc75253b-5d97-4da6-8fad-8fcd8245639a\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") "
Oct 30 21:52:58.122778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.122688    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\" (UniqueName: \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\") pod \"dc75253b-5d97-4da6-8fad-8fcd8245639a\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") "
Oct 30 21:52:58.122778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.122728    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\") pod \"dc75253b-5d97-4da6-8fad-8fcd8245639a\" (UID: \"dc75253b-5d97-4da6-8fad-8fcd8245639a\") "
Oct 30 21:52:58.123302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.123271    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca" pod "dc75253b-5d97-4da6-8fad-8fcd8245639a" (UID: "dc75253b-5d97-4da6-8fad-8fcd8245639a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:52:58.123898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.123871    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc75253b-5d97-4da6-8fad-8fcd8245639a-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:58.125370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.125347    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls" pod "dc75253b-5d97-4da6-8fad-8fcd8245639a" (UID: "dc75253b-5d97-4da6-8fad-8fcd8245639a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:58.125370 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.125356    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s" pod "dc75253b-5d97-4da6-8fad-8fcd8245639a" (UID: "dc75253b-5d97-4da6-8fad-8fcd8245639a"). InnerVolumeSpecName "kube-api-access-r256s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:52:58.129593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.129571    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs" pod "dc75253b-5d97-4da6-8fad-8fcd8245639a" (UID: "dc75253b-5d97-4da6-8fad-8fcd8245639a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:52:58.229514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.229005    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:58.229514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.229045    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r256s\" (UniqueName: \"kubernetes.io/projected/dc75253b-5d97-4da6-8fad-8fcd8245639a-kube-api-access-r256s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:58.229514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.229062    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc75253b-5d97-4da6-8fad-8fcd8245639a-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:52:58.497777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.497696    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:58.583646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.583616    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"
Oct 30 21:52:58.583869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.583647    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" event={"ID":"dc75253b-5d97-4da6-8fad-8fcd8245639a","Type":"ContainerDied","Data":"31fd7de48e3805f8a90955df11157e6af98ac4fc30e1997008be90e7533f8682"}
Oct 30 21:52:58.583869 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.583730    6738 scope.go:122] "RemoveContainer" containerID="4f69656f4f3b7fbe9a6f6392779043bd6044a068b0b2dd3cfd6dae3f6d39837e"
Oct 30 21:52:58.619568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.619513    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:52:58.624701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.624677    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44"]
Oct 30 21:52:58.645320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:58.645299    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:52:59.166067 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:59.166033    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" path="/var/lib/kubelet/pods/dc75253b-5d97-4da6-8fad-8fcd8245639a/volumes"
Oct 30 21:52:59.333395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:52:59.333340    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:00.068678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:00.068623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:00.845811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:00.845772    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8431de-575779cc9c-sfj44" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.152:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:53:00.846649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:00.846627    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:01.623213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:01.623178    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:02.532634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:02.532578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:03.350745 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:03.350714    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:04.151678 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:04.151635    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:04.946285 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:04.946249    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:05.084191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:05.084155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:05.279981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:05.279896    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:53:05.280247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:05.280213    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" containerID="cri-o://2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438" gracePeriod=30
Oct 30 21:53:05.471887 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:05.471854    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:06.317535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:06.317497    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:07.188263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:07.188226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:07.936509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:07.936479    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:08.295024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:08.294940    6738 patch_prober.go:29] interesting pod/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:53:08.295024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:08.294997    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:53:08.755669 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:08.755630    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:09.502852 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:09.502821    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:09.717274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:09.717239    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:53:09.782070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:09.781974    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.600590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.600562    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:53:10.688509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.688406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:10.708698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.708663    6738 generic.go:356] "Generic (PLEG): container finished" podID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerID="2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438" exitCode=0
Oct 30 21:53:10.709190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.709167    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" event={"ID":"2a1cf896-b64d-4adb-9da1-c0de35fa842c","Type":"ContainerDied","Data":"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"}
Oct 30 21:53:10.709360 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.709344    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp" event={"ID":"2a1cf896-b64d-4adb-9da1-c0de35fa842c","Type":"ContainerDied","Data":"4e77054ffb7523d714d69188037e7e213ddd067fc692eaaa389587216aae80e3"}
Oct 30 21:53:10.709533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.709504    6738 scope.go:122] "RemoveContainer" containerID="2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.721190    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722193    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722260    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722292    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722320    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722370    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722448    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722494    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722563    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.731459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.722803    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\") pod \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\" (UID: \"2a1cf896-b64d-4adb-9da1-c0de35fa842c\") "
Oct 30 21:53:10.740754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.735613    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:10.740754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.735931    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:10.740754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.737702    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:10.756527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.751318    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:10.757742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.757659    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:10.766026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.765991    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:10.772649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.772582    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:10.772649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.772594    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:53:10.772649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.772618    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q" pod "2a1cf896-b64d-4adb-9da1-c0de35fa842c" (UID: "2a1cf896-b64d-4adb-9da1-c0de35fa842c"). InnerVolumeSpecName "kube-api-access-vn49q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:10.809091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.809040    6738 scope.go:122] "RemoveContainer" containerID="2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"
Oct 30 21:53:10.809525 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:10.809497    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438\": container with ID starting with 2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438 not found: ID does not exist" containerID="2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"
Oct 30 21:53:10.809646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.809539    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438"} err="failed to get container status \"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438\": rpc error: code = NotFound desc = could not find container \"2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438\": container with ID starting with 2ab239b77e4364614803bd5142129114b1f7fe8cb1551222e55811fb3f77b438 not found: ID does not exist"
Oct 30 21:53:10.833288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833230    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833257    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833266    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833277    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833285    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833294    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833302    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vn49q\" (UniqueName: \"kubernetes.io/projected/2a1cf896-b64d-4adb-9da1-c0de35fa842c-kube-api-access-vn49q\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833310    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:10.833528 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:10.833318    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a1cf896-b64d-4adb-9da1-c0de35fa842c-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:11.047465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:11.047414    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:53:11.053666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:11.053641    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8431de-openshift-default-57bd47ddc7-nn9rp"]
Oct 30 21:53:11.167527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:11.167495    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" path="/var/lib/kubelet/pods/2a1cf896-b64d-4adb-9da1-c0de35fa842c/volumes"
Oct 30 21:53:11.449844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:11.449748    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:12.200785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:12.200751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:12.938427 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:12.938396    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:13.739041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:13.735194    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:14.533190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:14.533154    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:15.309991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:15.309944    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:16.057008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:16.056977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:16.179296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:16.179263    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:17.064533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:17.064505    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:18.035396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:18.035351    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:18.750627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:18.750592    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:19.423599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:19.423562    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:20.343799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:20.343527    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:20.464693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:20.464659    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:21.024225 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:21.024174    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:21.262098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:21.262001    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:21.883586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:21.883556    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:22.520768 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:22.520723    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:53:22.738282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:22.738254    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:23.527601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:23.527551    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:24.190783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:24.190748    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:24.764265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:24.764229    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" podUID="49f7d339-a578-4f54-b05c-3a8f5709b17d" containerName="main" containerID="cri-o://78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a" gracePeriod=45
Oct 30 21:53:24.961363 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:24.959719    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:25.107612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.107580    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:53:25.172641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172614    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\" (UniqueName: \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.172823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172730    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.172823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172760    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.172823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172814    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.173019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172840    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\" (UniqueName: \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.173019 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.172971    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\") pod \"49f7d339-a578-4f54-b05c-3a8f5709b17d\" (UID: \"49f7d339-a578-4f54-b05c-3a8f5709b17d\") "
Oct 30 21:53:25.173137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.173018    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:25.173196 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.173142    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:25.173262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.173239    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:25.173774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.173750    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.174119 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.174103    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.174231 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.174218    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.175769 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.175730    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:25.176186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.176164    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:25.176186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.176176    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp" pod "49f7d339-a578-4f54-b05c-3a8f5709b17d" (UID: "49f7d339-a578-4f54-b05c-3a8f5709b17d"). InnerVolumeSpecName "kube-api-access-tt6pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:25.278357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.278324    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt6pp\" (UniqueName: \"kubernetes.io/projected/49f7d339-a578-4f54-b05c-3a8f5709b17d-kube-api-access-tt6pp\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.278357 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.278359    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49f7d339-a578-4f54-b05c-3a8f5709b17d-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.278587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.278374    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49f7d339-a578-4f54-b05c-3a8f5709b17d-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:25.622671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.622627    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:25.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.874257    6738 generic.go:356] "Generic (PLEG): container finished" podID="49f7d339-a578-4f54-b05c-3a8f5709b17d" containerID="78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a" exitCode=0
Oct 30 21:53:25.874347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.874319    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"
Oct 30 21:53:25.874929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.874341    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" event={"ID":"49f7d339-a578-4f54-b05c-3a8f5709b17d","Type":"ContainerDied","Data":"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"}
Oct 30 21:53:25.874929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.874382    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp" event={"ID":"49f7d339-a578-4f54-b05c-3a8f5709b17d","Type":"ContainerDied","Data":"d33ba0276655110b57fad47107960c2203bc1b492e6198a4ff49a3141a5e5383"}
Oct 30 21:53:25.874929 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.874402    6738 scope.go:122] "RemoveContainer" containerID="78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"
Oct 30 21:53:25.887337 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.887322    6738 scope.go:122] "RemoveContainer" containerID="78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"
Oct 30 21:53:25.887664 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:25.887642    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a\": container with ID starting with 78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a not found: ID does not exist" containerID="78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"
Oct 30 21:53:25.887796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.887677    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a"} err="failed to get container status \"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a\": rpc error: code = NotFound desc = could not find container \"78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a\": container with ID starting with 78e5bc11b60b0c26ba301711ef5f9a6d9520617097de120deeed09f8396ea05a not found: ID does not exist"
Oct 30 21:53:25.899920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.899887    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:53:25.910570 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:25.910546    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-001667/test-model-001667-kserve-659c99f5f5-8jjsp"]
Oct 30 21:53:26.152386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.151255    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:53:26.152386 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.151543    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" podUID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" containerName="maas-api" containerID="cri-o://4c5254f6dcb50fe9782eedbdd019c836afcc5b5ffeaa1c7dd08caf36b6d09621" gracePeriod=30
Oct 30 21:53:26.205353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.205280    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.426515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.426487    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.570471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.561989    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.720576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.703135    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.892846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.892639    6738 generic.go:356] "Generic (PLEG): container finished" podID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" containerID="4c5254f6dcb50fe9782eedbdd019c836afcc5b5ffeaa1c7dd08caf36b6d09621" exitCode=0
Oct 30 21:53:26.892846 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.892881    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" event={"ID":"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a","Type":"ContainerDied","Data":"4c5254f6dcb50fe9782eedbdd019c836afcc5b5ffeaa1c7dd08caf36b6d09621"}
Oct 30 21:53:26.926455 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.922951    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:26.976611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:26.976548    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:53:27.053952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.053919    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\") pod \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") "
Oct 30 21:53:27.054121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.054015    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\") pod \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") "
Oct 30 21:53:27.054169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.054137    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\" (UniqueName: \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\") pod \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") "
Oct 30 21:53:27.054230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.054171    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\") pod \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\" (UID: \"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a\") "
Oct 30 21:53:27.054505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.054420    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca" pod "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" (UID: "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:27.055052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.055028    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:27.057504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.057225    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls" pod "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" (UID: "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:27.057504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.057358    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s" pod "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" (UID: "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a"). InnerVolumeSpecName "kube-api-access-2gp5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:27.057504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.057468    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs" pod "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" (UID: "eda11f2c-5e04-418b-ab69-54cb6b5b1c7a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:27.156372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.156336    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:27.156372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.156374    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gp5s\" (UniqueName: \"kubernetes.io/projected/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-kube-api-access-2gp5s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:27.156643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.156386    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:27.166203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.166171    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49f7d339-a578-4f54-b05c-3a8f5709b17d" path="/var/lib/kubelet/pods/49f7d339-a578-4f54-b05c-3a8f5709b17d/volumes"
Oct 30 21:53:27.517208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.517161    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:27.902822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.902794    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"
Oct 30 21:53:27.902822 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.902807    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx" event={"ID":"eda11f2c-5e04-418b-ab69-54cb6b5b1c7a","Type":"ContainerDied","Data":"1dde1a543143e0281036a1cc1d460557d85f9b895fb9b2cf329ca87c801a26fb"}
Oct 30 21:53:27.903361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.902878    6738 scope.go:122] "RemoveContainer" containerID="4c5254f6dcb50fe9782eedbdd019c836afcc5b5ffeaa1c7dd08caf36b6d09621"
Oct 30 21:53:27.921952 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.921927    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:53:27.930030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:27.930005    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3d9d7b-bbc45f5cc-bw9nx"]
Oct 30 21:53:28.229000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:28.228924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:28.406147 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:28.406111    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:29.145124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.145082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:29.216398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.216362    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" path="/var/lib/kubelet/pods/eda11f2c-5e04-418b-ab69-54cb6b5b1c7a/volumes"
Oct 30 21:53:29.694460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.694412    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:53:29.694993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.694963    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" podUID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" containerName="maas-api" containerID="cri-o://37e093a829d6aee28aafc9fce0694f48d5bcc94a5bb57d7ba731d53fec171c00" gracePeriod=30
Oct 30 21:53:29.767922 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.767759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:29.925267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.925047    6738 generic.go:356] "Generic (PLEG): container finished" podID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" containerID="37e093a829d6aee28aafc9fce0694f48d5bcc94a5bb57d7ba731d53fec171c00" exitCode=0
Oct 30 21:53:29.925267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.925172    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" event={"ID":"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e","Type":"ContainerDied","Data":"37e093a829d6aee28aafc9fce0694f48d5bcc94a5bb57d7ba731d53fec171c00"}
Oct 30 21:53:30.010539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:29.991046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.176655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.158501    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.323927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.323896    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:53:30.324969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.324945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.470044    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\") pod \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") "
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.470121    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\") pod \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") "
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.470142    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\") pod \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") "
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.470382    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\" (UniqueName: \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\") pod \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\" (UID: \"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e\") "
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.471346    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca" pod "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" (UID: "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:30.474510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.472384    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:30.480888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.480824    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls" pod "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" (UID: "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:30.482530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.482500    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq" pod "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" (UID: "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e"). InnerVolumeSpecName "kube-api-access-85bnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:30.483632 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.483601    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs" pod "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" (UID: "a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:30.574044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.573962    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:30.574044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.573988    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:30.574044 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.573998    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85bnq\" (UniqueName: \"kubernetes.io/projected/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e-kube-api-access-85bnq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:30.713648 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.713590    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:30.941747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.941717    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"
Oct 30 21:53:30.941747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.941738    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k" event={"ID":"a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e","Type":"ContainerDied","Data":"926e8d69f20b3bfab140d8dae41f1ea02fd7668f155f2ed1a8519393bf1456cf"}
Oct 30 21:53:30.941970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.941806    6738 scope.go:122] "RemoveContainer" containerID="37e093a829d6aee28aafc9fce0694f48d5bcc94a5bb57d7ba731d53fec171c00"
Oct 30 21:53:30.972583 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.972557    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:53:30.976495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:30.976462    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-570f01-686c544888-nkp7k"]
Oct 30 21:53:31.170239 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:31.170202    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" path="/var/lib/kubelet/pods/a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e/volumes"
Oct 30 21:53:31.193106 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:31.193036    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:32.037134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:32.037095    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:32.846362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:32.846328    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.417637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.417603    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.545684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.545653    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.673543 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.673448    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:33.740215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.740177    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:53:33.740597 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.740564    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy" containerID="cri-o://56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6" gracePeriod=30
Oct 30 21:53:33.806421 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:33.806391    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:34.644276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:34.644193    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:35.613245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:35.613216    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:36.293499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:36.293466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:36.983819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:36.983782    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:37.148991 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:37.148960    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:37.280661 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:37.280582    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:37.687325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:37.687297    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:38.085303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:38.085223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:39.055483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.055420    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:53:39.059780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.059736    6738 generic.go:356] "Generic (PLEG): container finished" podID="817ed678-747a-494d-979e-10c0170ef7ca" containerID="56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6" exitCode=0
Oct 30 21:53:39.059919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.059794    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"
Oct 30 21:53:39.059919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.059821    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerDied","Data":"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"}
Oct 30 21:53:39.060006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.059926    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6" event={"ID":"817ed678-747a-494d-979e-10c0170ef7ca","Type":"ContainerDied","Data":"99451d54a9a826f743fc60af8774b7d2ee9467cf222f3254d6e723698ac30447"}
Oct 30 21:53:39.060006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.059958    6738 scope.go:122] "RemoveContainer" containerID="56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"
Oct 30 21:53:39.071198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.071180    6738 scope.go:122] "RemoveContainer" containerID="e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"
Oct 30 21:53:39.082605 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.082585    6738 scope.go:122] "RemoveContainer" containerID="56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"
Oct 30 21:53:39.083038 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:39.083012    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6\": container with ID starting with 56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6 not found: ID does not exist" containerID="56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"
Oct 30 21:53:39.083182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.083049    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6"} err="failed to get container status \"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6\": rpc error: code = NotFound desc = could not find container \"56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6\": container with ID starting with 56abc945088f4dd7e20881bd16f81a2e68b3c1d61ef9271d6cac5b0e6cea12c6 not found: ID does not exist"
Oct 30 21:53:39.083182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.083073    6738 scope.go:122] "RemoveContainer" containerID="e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"
Oct 30 21:53:39.083416 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:39.083386    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5\": container with ID starting with e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5 not found: ID does not exist" containerID="e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"
Oct 30 21:53:39.083939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.083430    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5"} err="failed to get container status \"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5\": rpc error: code = NotFound desc = could not find container \"e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5\": container with ID starting with e49fe1800155c76bac1a8dcb56719cf0c95f395d0dc56739f6a75f35ce26ebb5 not found: ID does not exist"
Oct 30 21:53:39.140614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.140571    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:39.174976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.174258    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175111    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175179    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175253    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175337    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175367    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175508    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175574 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175554    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.175721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.175595    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\") pod \"817ed678-747a-494d-979e-10c0170ef7ca\" (UID: \"817ed678-747a-494d-979e-10c0170ef7ca\") "
Oct 30 21:53:39.210515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.205759    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:39.210515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.206129    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:39.210515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.206362    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:39.210515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.206576    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:39.222025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.221981    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:39.223057 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.223023    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:53:39.223224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.223049    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:53:39.227475 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.225149    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "kube-api-access-nbjss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:39.227634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.227603    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert" pod "817ed678-747a-494d-979e-10c0170ef7ca" (UID: "817ed678-747a-494d-979e-10c0170ef7ca"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:39.278546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278472    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/817ed678-747a-494d-979e-10c0170ef7ca-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278506    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278546 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278521    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/817ed678-747a-494d-979e-10c0170ef7ca-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278570    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278601    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278620    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278637    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/817ed678-747a-494d-979e-10c0170ef7ca-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278651    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.278727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.278668    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbjss\" (UniqueName: \"kubernetes.io/projected/817ed678-747a-494d-979e-10c0170ef7ca-kube-api-access-nbjss\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:39.396066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.396034    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:53:39.409523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.409492    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3d9d7b-openshift-default-78bcf5fcc-ccxm6"]
Oct 30 21:53:39.594511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:39.594378    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:40.273524 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:40.273485    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:41.012935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:41.012900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:41.170886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:41.170848    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="817ed678-747a-494d-979e-10c0170ef7ca" path="/var/lib/kubelet/pods/817ed678-747a-494d-979e-10c0170ef7ca/volumes"
Oct 30 21:53:41.858901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:41.858871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:42.621499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:42.621454    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:43.478085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:43.478021    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:44.167540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:44.167508    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:44.486776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:44.486680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:44.601945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:44.601911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:45.227950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:45.227915    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:45.896494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:45.896459    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:46.740875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:46.740802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:47.435715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:47.435667    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.071079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:48.071041    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.264251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:48.260460    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:48.386267 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:48.386217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:49.022134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:49.022086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:49.756426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:49.756392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:50.168675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:50.168641    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:50.287585 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:50.287553    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:50.757946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:50.757916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:51.461556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.461520    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:53:51.461958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.461785    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerName="maas-api" containerID="cri-o://e39604a6993d8dca1faa58786d6823104bf067e4956c6b20f5d3fab172df66d2" gracePeriod=30
Oct 30 21:53:51.534134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.534081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:51.578939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.578902    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.160:8443/health\": dial tcp 10.134.0.160:8443: connect: connection refused"
Oct 30 21:53:51.800695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.800610    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:51.975102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:51.975002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:52.224333 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.224295    6738 generic.go:356] "Generic (PLEG): container finished" podID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerID="e39604a6993d8dca1faa58786d6823104bf067e4956c6b20f5d3fab172df66d2" exitCode=0
Oct 30 21:53:52.224742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.224719    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" event={"ID":"f3ba5b05-aeac-4956-8360-f688e2aa8d55","Type":"ContainerDied","Data":"e39604a6993d8dca1faa58786d6823104bf067e4956c6b20f5d3fab172df66d2"}
Oct 30 21:53:52.283312 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.283285    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.310572    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") pod \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") "
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.310651    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\") pod \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") "
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.310732    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\") pod \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") "
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.310778    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\" (UniqueName: \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\") pod \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\" (UID: \"f3ba5b05-aeac-4956-8360-f688e2aa8d55\") "
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.311494    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca" pod "f3ba5b05-aeac-4956-8360-f688e2aa8d55" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:53:52.312460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.312108    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3ba5b05-aeac-4956-8360-f688e2aa8d55-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:52.322525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.318930    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs" pod "f3ba5b05-aeac-4956-8360-f688e2aa8d55" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:52.322525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.319315    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls" pod "f3ba5b05-aeac-4956-8360-f688e2aa8d55" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:53:52.334482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.328600    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx" pod "f3ba5b05-aeac-4956-8360-f688e2aa8d55" (UID: "f3ba5b05-aeac-4956-8360-f688e2aa8d55"). InnerVolumeSpecName "kube-api-access-cg6gx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:53:52.343531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.342544    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:52.413497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.413418    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:52.413497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.413495    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3ba5b05-aeac-4956-8360-f688e2aa8d55-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:52.413497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.413510    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg6gx\" (UniqueName: \"kubernetes.io/projected/f3ba5b05-aeac-4956-8360-f688e2aa8d55-kube-api-access-cg6gx\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:53:52.485224 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:52.485113    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:52.639590 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:53:52.639560    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:53:53.233547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.233517    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"
Oct 30 21:53:53.233719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.233554    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp" event={"ID":"f3ba5b05-aeac-4956-8360-f688e2aa8d55","Type":"ContainerDied","Data":"199985a08c41bc472a31a2931b0afe96d396aaf59652222489c91c3529365acb"}
Oct 30 21:53:53.233719 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.233616    6738 scope.go:122] "RemoveContainer" containerID="e39604a6993d8dca1faa58786d6823104bf067e4956c6b20f5d3fab172df66d2"
Oct 30 21:53:53.261927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.261898    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:53:53.265514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.265492    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-cf51d4-d79d4856f-g4pbp"]
Oct 30 21:53:53.425653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.425557    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:53.960576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:53.960534    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:54.170550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:54.167490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:54.316945 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:54.316843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:54.541272 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:54.541238    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:54.913233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:54.913197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:55.167179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:55.167101    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" path="/var/lib/kubelet/pods/f3ba5b05-aeac-4956-8360-f688e2aa8d55/volumes"
Oct 30 21:53:55.346104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:55.346068    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.082307 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:56.082204    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:56.850571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:56.850543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:57.545780 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:57.545751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:58.243995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:58.243958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.005730 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.005696    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.208139 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.208101    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.343023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.342939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.488305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.488273    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:53:59.488821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.488789    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" containerID="cri-o://89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b" gracePeriod=30
Oct 30 21:53:59.556919 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.556885    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:53:59.625515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.611806    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:53:59.625515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.612113    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" podUID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" containerName="maas-api" containerID="cri-o://e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21" gracePeriod=30
Oct 30 21:53:59.902420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:53:59.902294    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:00.256491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.256312    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:00.326926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.326902    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:54:00.341596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.341564    6738 generic.go:356] "Generic (PLEG): container finished" podID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" containerID="e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21" exitCode=0
Oct 30 21:54:00.341724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.341600    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" event={"ID":"23a8f0b7-2960-4dad-a3f6-dc884a011bbe","Type":"ContainerDied","Data":"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"}
Oct 30 21:54:00.341724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.341623    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"
Oct 30 21:54:00.341724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.341653    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2" event={"ID":"23a8f0b7-2960-4dad-a3f6-dc884a011bbe","Type":"ContainerDied","Data":"7cd024e0ad5565749b0c868c246d487f64319fbd52ed9d0b9900b29ac7982b6e"}
Oct 30 21:54:00.341724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.341675    6738 scope.go:122] "RemoveContainer" containerID="e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"
Oct 30 21:54:00.354819 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.354800    6738 scope.go:122] "RemoveContainer" containerID="e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"
Oct 30 21:54:00.355131 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:00.355090    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21\": container with ID starting with e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21 not found: ID does not exist" containerID="e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"
Oct 30 21:54:00.355187 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.355123    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21"} err="failed to get container status \"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21\": rpc error: code = NotFound desc = could not find container \"e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21\": container with ID starting with e6b333951bf7cb9ed8b2ace82eedaefd69e46447acbfe621e47cbbebcabc0e21 not found: ID does not exist"
Oct 30 21:54:00.406413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.406148    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:00.432009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.431934    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\") pod \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") "
Oct 30 21:54:00.432136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.432086    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\" (UniqueName: \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\") pod \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") "
Oct 30 21:54:00.432195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.432145    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\") pod \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") "
Oct 30 21:54:00.432280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.432265    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") pod \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\" (UID: \"23a8f0b7-2960-4dad-a3f6-dc884a011bbe\") "
Oct 30 21:54:00.432347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.432292    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca" pod "23a8f0b7-2960-4dad-a3f6-dc884a011bbe" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:00.433169 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.433093    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:00.435112 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.435013    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls" pod "23a8f0b7-2960-4dad-a3f6-dc884a011bbe" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:00.435479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.435417    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs" pod "23a8f0b7-2960-4dad-a3f6-dc884a011bbe" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:00.435479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.435470    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74" pod "23a8f0b7-2960-4dad-a3f6-dc884a011bbe" (UID: "23a8f0b7-2960-4dad-a3f6-dc884a011bbe"). InnerVolumeSpecName "kube-api-access-6dm74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:00.538579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.535340    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:00.538579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.535425    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:00.538579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.535492    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dm74\" (UniqueName: \"kubernetes.io/projected/23a8f0b7-2960-4dad-a3f6-dc884a011bbe-kube-api-access-6dm74\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:00.556770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.539088    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:00.672476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.672428    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:54:00.680913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.680884    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-e596df-5dd4b497-vspv2"]
Oct 30 21:54:00.908599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:00.908562    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:01.166488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:01.166411    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" path="/var/lib/kubelet/pods/23a8f0b7-2960-4dad-a3f6-dc884a011bbe/volumes"
Oct 30 21:54:01.174741 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:01.174707    6738 patch_prober.go:29] interesting pod/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:54:01.174883 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:01.174786    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:54:01.281261 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:01.281226    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:01.919341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:01.919310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.032078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:02.032042    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.336967 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:02.336876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:02.477893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:02.477860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:03.350838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:03.350807    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:04.215459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.206517    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:04.806318 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.806291    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:54:04.841103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841064    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841132    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841166    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841289 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841242    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841353    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841488 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841393    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841510    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841576    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.841726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841616    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:04.841973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.841949    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:04.842117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.842097    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:04.842513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.842487    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:04.842816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.842679    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\") pod \"b0b8f042-e5c6-41c1-805e-63835541c2f3\" (UID: \"b0b8f042-e5c6-41c1-805e-63835541c2f3\") "
Oct 30 21:54:04.844271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.843615    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.844271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.843640    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.844271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.843654    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b8f042-e5c6-41c1-805e-63835541c2f3-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.844271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.843668    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.844651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.844618    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:04.848925 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.848903    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:04.849483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.849428    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:04.849993 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.849971    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "kube-api-access-n758k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:04.851059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.850918    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token" pod "b0b8f042-e5c6-41c1-805e-63835541c2f3" (UID: "b0b8f042-e5c6-41c1-805e-63835541c2f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:04.945256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.945222    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.945256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.945249    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.945256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.945258    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n758k\" (UniqueName: \"kubernetes.io/projected/b0b8f042-e5c6-41c1-805e-63835541c2f3-kube-api-access-n758k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.945523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.945268    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:04.945523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:04.945276    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b8f042-e5c6-41c1-805e-63835541c2f3-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:05.161484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.161423    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:05.426516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.426402    6738 generic.go:356] "Generic (PLEG): container finished" podID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerID="89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b" exitCode=0
Oct 30 21:54:05.426516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.426468    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"
Oct 30 21:54:05.426516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.426489    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" event={"ID":"b0b8f042-e5c6-41c1-805e-63835541c2f3","Type":"ContainerDied","Data":"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"}
Oct 30 21:54:05.426764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.426549    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54" event={"ID":"b0b8f042-e5c6-41c1-805e-63835541c2f3","Type":"ContainerDied","Data":"26c9c67fc8de655955ab6bb2facc4c4e878269b81dfa09726ed473c9f1cd42fe"}
Oct 30 21:54:05.426764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.426572    6738 scope.go:122] "RemoveContainer" containerID="89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"
Oct 30 21:54:05.438913 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.438891    6738 scope.go:122] "RemoveContainer" containerID="89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"
Oct 30 21:54:05.439185 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:05.439167    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b\": container with ID starting with 89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b not found: ID does not exist" containerID="89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"
Oct 30 21:54:05.439253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.439198    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b"} err="failed to get container status \"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b\": rpc error: code = NotFound desc = could not find container \"89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b\": container with ID starting with 89c3fff18bef7e781667448e1f01f2be9b0aee29826870a4f5dec0050490b91b not found: ID does not exist"
Oct 30 21:54:05.465076 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.465053    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:54:05.472657 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:05.472630    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-cf51d4-openshift-default-5fbcd5645c-fsg54"]
Oct 30 21:54:06.159851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:06.159823    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:06.933596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:06.933558    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:07.166096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:07.166064    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" path="/var/lib/kubelet/pods/b0b8f042-e5c6-41c1-805e-63835541c2f3/volumes"
Oct 30 21:54:07.595864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:07.595831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:07.729848 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:07.729813    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.016523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:08.016489    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:08.666422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:08.666380    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:09.625671 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:09.625637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:10.553302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:10.553267    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:54:10.600066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:10.599977    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:10.731641 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:10.731604    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:11.352118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:11.352087    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:12.148109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:12.148076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:12.895419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:12.895383    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:54:12.913252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:12.913220    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:54:12.945198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:12.945170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:13.054774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:13.054711    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:13.719527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:13.719490    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:14.402843 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:14.402804    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.204537 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:15.204502    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:15.554550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:15.554470    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing" containerID="cri-o://7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787" gracePeriod=25
Oct 30 21:54:16.053226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:16.053197    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:16.898153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:16.898121    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:17.616046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:17.615986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:17.896013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:17.895905    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing" containerID="cri-o://6cfe70996e580afa99cc283d8f339587b93e09f87506cf3456c52e7350b514b1" gracePeriod=25
Oct 30 21:54:17.914482 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:17.914449    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing" containerID="cri-o://47a183e1f6c282233e7f348a7e77a84e0808e9bf65df9671f5fb5731d62d7738" gracePeriod=25
Oct 30 21:54:18.289850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:18.289705    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:18.671912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:18.671875    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:18.802003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:18.801943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:18.987210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:18.987119    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.385884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:19.385851    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.509505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:19.509467    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.621854 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:19.621817    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:19.961110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:19.961076    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.248646 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:20.248565    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:20.294425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:20.294392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.464465 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:20.463986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.726271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:20.726237    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:20.842091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:20.842058    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:21.074775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:21.074686    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:21.234790 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:21.234752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:21.551567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:21.551533    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:22.254418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:22.254382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:22.523112 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:22.523024    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:54:23.053511 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:23.053470    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:23.743334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:23.743301    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:24.538613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:24.538577    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:25.066504 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:25.066425    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:25.066698 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:25.066516    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:25.251300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:25.251271    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:25.999034 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:25.999002    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.152137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:26.152107    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:26.417206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:26.417176    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:27.163977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:27.163949    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.092898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:28.092853    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.212410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:28.212376    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:28.436960 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:28.436920    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:54:28.437228 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:28.437191    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy" containerID="cri-o://64db387be3c83cfab91aaec91fb2c21bdf5b7f83f87123884a901a0a62d64f5f" gracePeriod=30
Oct 30 21:54:28.534909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:28.534871    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:29.281513 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:29.281468    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:30.154729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:30.154698    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:30.248892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:30.248839    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:31.202188 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:31.202156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:32.050136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:32.050038    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:32.922628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:32.922593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.553523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.553485    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:54:33.554013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.553984    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" podUID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" containerName="maas-api" containerID="cri-o://7e596771d5579006e1f6bd0efe57b54c550eb41a6d5ed03bfbe70a7efbb57b2e" gracePeriod=30
Oct 30 21:54:33.621164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.621099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.891882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.877859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:33.937457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.934958    6738 generic.go:356] "Generic (PLEG): container finished" podID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerID="64db387be3c83cfab91aaec91fb2c21bdf5b7f83f87123884a901a0a62d64f5f" exitCode=0
Oct 30 21:54:33.937457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:33.935174    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" event={"ID":"bc09f4b3-0436-41fa-ad66-86721bcee130","Type":"ContainerDied","Data":"64db387be3c83cfab91aaec91fb2c21bdf5b7f83f87123884a901a0a62d64f5f"}
Oct 30 21:54:34.057977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.056631    6738 generic.go:356] "Generic (PLEG): container finished" podID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" containerID="7e596771d5579006e1f6bd0efe57b54c550eb41a6d5ed03bfbe70a7efbb57b2e" exitCode=0
Oct 30 21:54:34.057977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.056773    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" event={"ID":"9efd6bd5-49f0-44e3-9805-c8e437cc4951","Type":"ContainerDied","Data":"7e596771d5579006e1f6bd0efe57b54c550eb41a6d5ed03bfbe70a7efbb57b2e"}
Oct 30 21:54:34.095515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.092026    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:54:34.165757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.165732    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:54:34.221608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.221547    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:34.247695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247664    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.247884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247700    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.247884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247769    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\") pod \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") "
Oct 30 21:54:34.247884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247791    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.247884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247833    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.247884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247865    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\" (UniqueName: \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\") pod \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") "
Oct 30 21:54:34.248178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.247945    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.248178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248002    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.248178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248103    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.248178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248159    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.248178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248166    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248171    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca" pod "9efd6bd5-49f0-44e3-9805-c8e437cc4951" (UID: "9efd6bd5-49f0-44e3-9805-c8e437cc4951"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248336    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248384    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\") pod \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") "
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248424    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\") pod \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\" (UID: \"9efd6bd5-49f0-44e3-9805-c8e437cc4951\") "
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248475    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\") pod \"bc09f4b3-0436-41fa-ad66-86721bcee130\" (UID: \"bc09f4b3-0436-41fa-ad66-86721bcee130\") "
Oct 30 21:54:34.248498 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248427    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:34.248872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.248520    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:34.249110 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249084    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:34.249578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249557    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.249658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249583    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.249658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249599    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.249658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249611    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.249658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249625    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bc09f4b3-0436-41fa-ad66-86721bcee130-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.249658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.249638    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9efd6bd5-49f0-44e3-9805-c8e437cc4951-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.251762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.251720    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:34.251871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.251781    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd" pod "9efd6bd5-49f0-44e3-9805-c8e437cc4951" (UID: "9efd6bd5-49f0-44e3-9805-c8e437cc4951"). InnerVolumeSpecName "kube-api-access-9z5sd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:34.251939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.251884    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "kube-api-access-2pns6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:34.251939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.251921    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:34.251939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.251931    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs" pod "9efd6bd5-49f0-44e3-9805-c8e437cc4951" (UID: "9efd6bd5-49f0-44e3-9805-c8e437cc4951"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:34.252152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.252130    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy" pod "bc09f4b3-0436-41fa-ad66-86721bcee130" (UID: "bc09f4b3-0436-41fa-ad66-86721bcee130"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:34.253512 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.253450    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls" pod "9efd6bd5-49f0-44e3-9805-c8e437cc4951" (UID: "9efd6bd5-49f0-44e3-9805-c8e437cc4951"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:34.351461 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.342452    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354299    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354334    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354348    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9efd6bd5-49f0-44e3-9805-c8e437cc4951-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354363    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354377    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354391    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pns6\" (UniqueName: \"kubernetes.io/projected/bc09f4b3-0436-41fa-ad66-86721bcee130-kube-api-access-2pns6\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.358459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.354405    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z5sd\" (UniqueName: \"kubernetes.io/projected/9efd6bd5-49f0-44e3-9805-c8e437cc4951-kube-api-access-9z5sd\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:34.702551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:34.702514    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.065832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.065790    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"
Oct 30 21:54:35.065832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.065825    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn" event={"ID":"9efd6bd5-49f0-44e3-9805-c8e437cc4951","Type":"ContainerDied","Data":"2410eaa0ccac9f5551a5c32044032d176649b13ef307876683c1f17a67f1a2db"}
Oct 30 21:54:35.066344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.065883    6738 scope.go:122] "RemoveContainer" containerID="7e596771d5579006e1f6bd0efe57b54c550eb41a6d5ed03bfbe70a7efbb57b2e"
Oct 30 21:54:35.066668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.066635    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:35.066820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.066700    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:35.067948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.067925    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt" event={"ID":"bc09f4b3-0436-41fa-ad66-86721bcee130","Type":"ContainerDied","Data":"259cbb5f86f4d6452319075fb35d50c3ec19721b8d05c67a2c2b8e0d0eab93c1"}
Oct 30 21:54:35.067948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.067940    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"
Oct 30 21:54:35.078122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.078104    6738 scope.go:122] "RemoveContainer" containerID="64db387be3c83cfab91aaec91fb2c21bdf5b7f83f87123884a901a0a62d64f5f"
Oct 30 21:54:35.099294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.099256    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:54:35.103136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.103115    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f1ed97-556ddc8697-z4xdn"]
Oct 30 21:54:35.116070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.116045    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:54:35.119396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.119376    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-8072a0-openshift-default-7d75695589-dk9vt"]
Oct 30 21:54:35.167317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.167290    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" path="/var/lib/kubelet/pods/9efd6bd5-49f0-44e3-9805-c8e437cc4951/volumes"
Oct 30 21:54:35.167851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.167829    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" path="/var/lib/kubelet/pods/bc09f4b3-0436-41fa-ad66-86721bcee130/volumes"
Oct 30 21:54:35.284464 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.279722    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:35.852060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.852033    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:54:35.977467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.977419    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\" (UniqueName: \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\") pod \"724f2199-282c-48c3-8e22-0075d8543f7a\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") "
Oct 30 21:54:35.977687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.977654    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\") pod \"724f2199-282c-48c3-8e22-0075d8543f7a\" (UID: \"724f2199-282c-48c3-8e22-0075d8543f7a\") "
Oct 30 21:54:35.978033 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.978010    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume" pod "724f2199-282c-48c3-8e22-0075d8543f7a" (UID: "724f2199-282c-48c3-8e22-0075d8543f7a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:35.978165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.978149    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/724f2199-282c-48c3-8e22-0075d8543f7a-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:35.979792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:35.979747    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm" pod "724f2199-282c-48c3-8e22-0075d8543f7a" (UID: "724f2199-282c-48c3-8e22-0075d8543f7a"). InnerVolumeSpecName "kube-api-access-784xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:36.078212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.078131    6738 generic.go:356] "Generic (PLEG): container finished" podID="724f2199-282c-48c3-8e22-0075d8543f7a" containerID="7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787" exitCode=0
Oct 30 21:54:36.078212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.078184    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"
Oct 30 21:54:36.078752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.078216    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" event={"ID":"724f2199-282c-48c3-8e22-0075d8543f7a","Type":"ContainerDied","Data":"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"}
Oct 30 21:54:36.078752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.078264    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x" event={"ID":"724f2199-282c-48c3-8e22-0075d8543f7a","Type":"ContainerDied","Data":"ecc208b319cccec0ca10a212b091dd86f7bb692be2c6bcec0170fdad14b6b4e5"}
Oct 30 21:54:36.078752 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.078296    6738 scope.go:122] "RemoveContainer" containerID="7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"
Oct 30 21:54:36.079179 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.079153    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-784xm\" (UniqueName: \"kubernetes.io/projected/724f2199-282c-48c3-8e22-0075d8543f7a-kube-api-access-784xm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:36.090833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.090818    6738 scope.go:122] "RemoveContainer" containerID="7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"
Oct 30 21:54:36.091153 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:36.091132    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787\": container with ID starting with 7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787 not found: ID does not exist" containerID="7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"
Oct 30 21:54:36.091233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.091174    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787"} err="failed to get container status \"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787\": rpc error: code = NotFound desc = could not find container \"7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787\": container with ID starting with 7e6692afe7d162f8da2835d916dd0ab6109c649f1077432ba9cd0cc9670ee787 not found: ID does not exist"
Oct 30 21:54:36.103283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.103256    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:54:36.110841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.110820    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-87ae68-55cc5fc669-gl57x"]
Oct 30 21:54:36.219760 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.219716    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:36.982178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:36.982134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.054683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.054654    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:54:37.062718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.062681    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:54:37.148137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.148101    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.170162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.170128    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" path="/var/lib/kubelet/pods/724f2199-282c-48c3-8e22-0075d8543f7a/volumes"
Oct 30 21:54:37.377458 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.374578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:37.938060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:37.938019    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:38.110495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.110268    6738 generic.go:356] "Generic (PLEG): container finished" podID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerID="6cfe70996e580afa99cc283d8f339587b93e09f87506cf3456c52e7350b514b1" exitCode=0
Oct 30 21:54:38.110495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.110315    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" event={"ID":"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836","Type":"ContainerDied","Data":"6cfe70996e580afa99cc283d8f339587b93e09f87506cf3456c52e7350b514b1"}
Oct 30 21:54:38.114750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.112494    6738 generic.go:356] "Generic (PLEG): container finished" podID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerID="47a183e1f6c282233e7f348a7e77a84e0808e9bf65df9671f5fb5731d62d7738" exitCode=0
Oct 30 21:54:38.114750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.112579    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" event={"ID":"ad50ef83-4035-47e7-a027-b1e6395290a2","Type":"ContainerDied","Data":"47a183e1f6c282233e7f348a7e77a84e0808e9bf65df9671f5fb5731d62d7738"}
Oct 30 21:54:38.215274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.215250    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:54:38.239477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.237765    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\" (UniqueName: \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\") pod \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") "
Oct 30 21:54:38.239477 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.238016    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\") pod \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\" (UID: \"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836\") "
Oct 30 21:54:38.239652 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.239532    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume" pod "3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" (UID: "3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:38.240522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.240273    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.241226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.241205    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz" pod "3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" (UID: "3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836"). InnerVolumeSpecName "kube-api-access-ssfpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:38.250805 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.250781    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:54:38.340860 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.340826    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\") pod \"ad50ef83-4035-47e7-a027-b1e6395290a2\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") "
Oct 30 21:54:38.341047 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.340946    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\" (UniqueName: \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\") pod \"ad50ef83-4035-47e7-a027-b1e6395290a2\" (UID: \"ad50ef83-4035-47e7-a027-b1e6395290a2\") "
Oct 30 21:54:38.341185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.341162    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume" pod "ad50ef83-4035-47e7-a027-b1e6395290a2" (UID: "ad50ef83-4035-47e7-a027-b1e6395290a2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:38.341520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.341501    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssfpz\" (UniqueName: \"kubernetes.io/projected/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836-kube-api-access-ssfpz\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.341604 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.341523    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad50ef83-4035-47e7-a027-b1e6395290a2-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.343248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.343227    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8" pod "ad50ef83-4035-47e7-a027-b1e6395290a2" (UID: "ad50ef83-4035-47e7-a027-b1e6395290a2"). InnerVolumeSpecName "kube-api-access-pcwg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:38.443535 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.443459    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcwg8\" (UniqueName: \"kubernetes.io/projected/ad50ef83-4035-47e7-a027-b1e6395290a2-kube-api-access-pcwg8\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:38.675164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:38.675132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:39.081619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.081587    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:54:39.134230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.134203    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"
Oct 30 21:54:39.135980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.134228    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp" event={"ID":"3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836","Type":"ContainerDied","Data":"0f46a52e1f1e81947929710e1cfbf2b42f48b9b82d45fc9992d312c9595bbcdc"}
Oct 30 21:54:39.136161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.136020    6738 scope.go:122] "RemoveContainer" containerID="6cfe70996e580afa99cc283d8f339587b93e09f87506cf3456c52e7350b514b1"
Oct 30 21:54:39.136857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.136818    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6" event={"ID":"ad50ef83-4035-47e7-a027-b1e6395290a2","Type":"ContainerDied","Data":"38f7a85972fb26425688c67cd2bc6fb0f21c151487ec7aaaa51abbb47cb21882"}
Oct 30 21:54:39.138271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.138254    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"
Oct 30 21:54:39.155563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.155539    6738 scope.go:122] "RemoveContainer" containerID="47a183e1f6c282233e7f348a7e77a84e0808e9bf65df9671f5fb5731d62d7738"
Oct 30 21:54:39.177530 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.177503    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:54:39.178968 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.178903    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-8072a0-5c85c96847-cvrsp"]
Oct 30 21:54:39.182747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.182724    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:39.198989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.198959    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:54:39.202325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.202305    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-8072a0-95c8476bc-vw6p6"]
Oct 30 21:54:39.292155 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.292123    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:39.734520 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:39.734487    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:40.443215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:40.443182    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.158098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.158056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.165279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.165253    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" path="/var/lib/kubelet/pods/3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836/volumes"
Oct 30 21:54:41.165817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.165801    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" path="/var/lib/kubelet/pods/ad50ef83-4035-47e7-a027-b1e6395290a2/volumes"
Oct 30 21:54:41.339251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.339213    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.461999 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.461902    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.774507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.770379    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:41.774507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.771829    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:54:41.774507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:41.772090    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerName="maas-api" containerID="cri-o://a194e624421c03d0fd1aa4342f4b7b70388825225bdc174f4cce0168e04426f0" gracePeriod=30
Oct 30 21:54:42.129515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.127410    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing" containerID="cri-o://7879c0c320bee35614ce816b22c97b6d8f7b88c080bb66fa81f7b44398c22d6e" gracePeriod=25
Oct 30 21:54:42.151685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.147275    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing" containerID="cri-o://3d2424709325858da11d145b6034c6398785639a5af4051e5816f63b980e5531" gracePeriod=25
Oct 30 21:54:42.200534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.172358    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.234721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.234429    6738 generic.go:356] "Generic (PLEG): container finished" podID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerID="a194e624421c03d0fd1aa4342f4b7b70388825225bdc174f4cce0168e04426f0" exitCode=0
Oct 30 21:54:42.234721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.234649    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" event={"ID":"e4405c11-ab67-41d8-81ee-0d9690c725bd","Type":"ContainerDied","Data":"a194e624421c03d0fd1aa4342f4b7b70388825225bdc174f4cce0168e04426f0"}
Oct 30 21:54:42.295626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.295587    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:54:42.347206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.347177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.432892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.432797    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\") pod \"e4405c11-ab67-41d8-81ee-0d9690c725bd\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") "
Oct 30 21:54:42.432892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.432858    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\") pod \"e4405c11-ab67-41d8-81ee-0d9690c725bd\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") "
Oct 30 21:54:42.433117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.432937    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\" (UniqueName: \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\") pod \"e4405c11-ab67-41d8-81ee-0d9690c725bd\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") "
Oct 30 21:54:42.433117 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.433026    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\") pod \"e4405c11-ab67-41d8-81ee-0d9690c725bd\" (UID: \"e4405c11-ab67-41d8-81ee-0d9690c725bd\") "
Oct 30 21:54:42.433505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.433474    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca" pod "e4405c11-ab67-41d8-81ee-0d9690c725bd" (UID: "e4405c11-ab67-41d8-81ee-0d9690c725bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:42.434090 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.434065    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4405c11-ab67-41d8-81ee-0d9690c725bd-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.435451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.435395    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs" pod "e4405c11-ab67-41d8-81ee-0d9690c725bd" (UID: "e4405c11-ab67-41d8-81ee-0d9690c725bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:42.435548 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.435518    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls" pod "e4405c11-ab67-41d8-81ee-0d9690c725bd" (UID: "e4405c11-ab67-41d8-81ee-0d9690c725bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:54:42.435764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.435720    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2" pod "e4405c11-ab67-41d8-81ee-0d9690c725bd" (UID: "e4405c11-ab67-41d8-81ee-0d9690c725bd"). InnerVolumeSpecName "kube-api-access-9r6f2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:42.474738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.474709    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.551764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.550287    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.551764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.550330    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4405c11-ab67-41d8-81ee-0d9690c725bd-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.551764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.550348    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9r6f2\" (UniqueName: \"kubernetes.io/projected/e4405c11-ab67-41d8-81ee-0d9690c725bd-kube-api-access-9r6f2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:42.618203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.618171    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:42.799584 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.799498    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:54:42.799777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.799706    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" containerID="cri-o://4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332" gracePeriod=30
Oct 30 21:54:42.948549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:42.948507    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:43.243545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.243508    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"
Oct 30 21:54:43.243545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.243523    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" event={"ID":"e4405c11-ab67-41d8-81ee-0d9690c725bd","Type":"ContainerDied","Data":"e16c004e3c2f49d1320bce4febef83bd44b4fdab92b9b43603d51d83cdd0ffe2"}
Oct 30 21:54:43.243824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.243577    6738 scope.go:122] "RemoveContainer" containerID="a194e624421c03d0fd1aa4342f4b7b70388825225bdc174f4cce0168e04426f0"
Oct 30 21:54:43.262750 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.262724    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:54:43.267054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.267032    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn"]
Oct 30 21:54:43.385391 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.385358    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:43.519091 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:43.518987    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:44.082599 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.082553    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" podUID="280474f1-ae85-4cbb-81e0-955a58c482b6" containerName="payload-processing" containerID="cri-o://f2a88b3baca4bafe0116dd074a09507411b24b0f60816f8cd48a90d5acf44e29" gracePeriod=25
Oct 30 21:54:44.253694 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.253656    6738 generic.go:356] "Generic (PLEG): container finished" podID="280474f1-ae85-4cbb-81e0-955a58c482b6" containerID="f2a88b3baca4bafe0116dd074a09507411b24b0f60816f8cd48a90d5acf44e29" exitCode=0
Oct 30 21:54:44.253866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.253738    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" event={"ID":"280474f1-ae85-4cbb-81e0-955a58c482b6","Type":"ContainerDied","Data":"f2a88b3baca4bafe0116dd074a09507411b24b0f60816f8cd48a90d5acf44e29"}
Oct 30 21:54:44.254579 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.254539    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:44.431800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.431778    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:54:44.489635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.489602    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\") pod \"280474f1-ae85-4cbb-81e0-955a58c482b6\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") "
Oct 30 21:54:44.490092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.490064    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume" pod "280474f1-ae85-4cbb-81e0-955a58c482b6" (UID: "280474f1-ae85-4cbb-81e0-955a58c482b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:44.490197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.490074    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\" (UniqueName: \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\") pod \"280474f1-ae85-4cbb-81e0-955a58c482b6\" (UID: \"280474f1-ae85-4cbb-81e0-955a58c482b6\") "
Oct 30 21:54:44.490726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.490699    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/280474f1-ae85-4cbb-81e0-955a58c482b6-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:44.492532 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.492501    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s" pod "280474f1-ae85-4cbb-81e0-955a58c482b6" (UID: "280474f1-ae85-4cbb-81e0-955a58c482b6"). InnerVolumeSpecName "kube-api-access-j8c7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:44.592007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:44.591981    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8c7s\" (UniqueName: \"kubernetes.io/projected/280474f1-ae85-4cbb-81e0-955a58c482b6-kube-api-access-j8c7s\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:45.130601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.130556    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-a378a2-6f56bb6966-8t4jn" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.122:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:54:45.131761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.131710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:45.203016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.202976    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" path="/var/lib/kubelet/pods/e4405c11-ab67-41d8-81ee-0d9690c725bd/volumes"
Oct 30 21:54:45.264707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.264670    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"
Oct 30 21:54:45.264912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.264722    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r" event={"ID":"280474f1-ae85-4cbb-81e0-955a58c482b6","Type":"ContainerDied","Data":"e9f49b3d6e9980ccd1e9730f84295698e77e43c1baef1de2e94e812635b9b9b1"}
Oct 30 21:54:45.264912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.264783    6738 scope.go:122] "RemoveContainer" containerID="f2a88b3baca4bafe0116dd074a09507411b24b0f60816f8cd48a90d5acf44e29"
Oct 30 21:54:45.282720 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.282696    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:54:45.286753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:45.286731    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-36a099-5786f6df69-kwr4r"]
Oct 30 21:54:46.063218 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:46.063185    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.670059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:46.670018    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:46.680029 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:46.679998    6738 patch_prober.go:29] interesting pod/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:54:46.680200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:46.680054    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:54:46.841024 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:46.840895    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.113494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:47.113463    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.165925 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:47.165890    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="280474f1-ae85-4cbb-81e0-955a58c482b6" path="/var/lib/kubelet/pods/280474f1-ae85-4cbb-81e0-955a58c482b6/volumes"
Oct 30 21:54:47.259039 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:47.259004    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:47.672414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:47.672380    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:47.701556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:47.701512    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.191367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.191334    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:54:48.263571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263475    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263624    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263706    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263758    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263766    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263832    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263898    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263928    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263952    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.263995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.263998    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\") pod \"b708ac00-44da-416f-bb61-291ab5148f74\" (UID: \"b708ac00-44da-416f-bb61-291ab5148f74\") "
Oct 30 21:54:48.264619 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.264597    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.264839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.264735    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:48.265131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.265105    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:48.265302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.265224    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:48.266775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.266710    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:48.267144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.267121    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:48.267338 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.267315    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:48.277847 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.277814    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:48.283251 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.283199    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr" pod "b708ac00-44da-416f-bb61-291ab5148f74" (UID: "b708ac00-44da-416f-bb61-291ab5148f74"). InnerVolumeSpecName "kube-api-access-v7kgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:48.305871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.305841    6738 generic.go:356] "Generic (PLEG): container finished" podID="b708ac00-44da-416f-bb61-291ab5148f74" containerID="4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332" exitCode=0
Oct 30 21:54:48.306006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.305889    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"
Oct 30 21:54:48.306006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.305897    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" event={"ID":"b708ac00-44da-416f-bb61-291ab5148f74","Type":"ContainerDied","Data":"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"}
Oct 30 21:54:48.306006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.305967    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2" event={"ID":"b708ac00-44da-416f-bb61-291ab5148f74","Type":"ContainerDied","Data":"a4036ad40e2d21be07aacbac931b44e188fecb125f128191c2b855d4af781c49"}
Oct 30 21:54:48.306006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.305992    6738 scope.go:122] "RemoveContainer" containerID="4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"
Oct 30 21:54:48.323481 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.323422    6738 scope.go:122] "RemoveContainer" containerID="4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"
Oct 30 21:54:48.323852 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:48.323827    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332\": container with ID starting with 4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332 not found: ID does not exist" containerID="4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"
Oct 30 21:54:48.323928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.323862    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332"} err="failed to get container status \"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332\": rpc error: code = NotFound desc = could not find container \"4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332\": container with ID starting with 4567860a3b6497c914403e2a25f58c8de381b8ca90ed181c394fcce1ebac0332 not found: ID does not exist"
Oct 30 21:54:48.341542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.341515    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:54:48.349361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.349332    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-05a342-openshift-default-6cd998c4ff-l42b2"]
Oct 30 21:54:48.366254 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366226    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366255    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366270    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7kgr\" (UniqueName: \"kubernetes.io/projected/b708ac00-44da-416f-bb61-291ab5148f74-kube-api-access-v7kgr\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366283    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366295    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b708ac00-44da-416f-bb61-291ab5148f74-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366302    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b708ac00-44da-416f-bb61-291ab5148f74-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366311    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.366362 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.366319    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b708ac00-44da-416f-bb61-291ab5148f74-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:48.408467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.393629    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:48.678166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:48.678130    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:49.172649 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:49.172611    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b708ac00-44da-416f-bb61-291ab5148f74" path="/var/lib/kubelet/pods/b708ac00-44da-416f-bb61-291ab5148f74/volumes"
Oct 30 21:54:49.331928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:49.331892    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:50.193313 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:50.193278    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:50.345182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:50.345150    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:50.569562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:50.566541    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:54:50.569562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:50.566792    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy" containerID="cri-o://2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705" gracePeriod=30
Oct 30 21:54:50.602909 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:50.602876    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:51.414943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:51.414911    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:52.248754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.248719    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:52.477132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.477098    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:54:52.477588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.477346    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" containerID="cri-o://c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719" gracePeriod=30
Oct 30 21:54:52.480241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.480213    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:54:52.480474 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.480423    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy" containerID="cri-o://d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d" gracePeriod=30
Oct 30 21:54:52.516584 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:52.514357    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:54:52.757292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.757259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:52.887037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:52.887003    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:53.078137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:53.078105    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:54.169235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:54.169183    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:54.917663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:54.917623    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.431341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:55.431308    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.567920 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:55.567848    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.767166 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:55.767137    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:55.979409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:55.976498    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.026042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.026013    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:54:56.124199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124153    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124199 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124203    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124222    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124287    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124314    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124390    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124474    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124496    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.124759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.124526    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\") pod \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\" (UID: \"a5b54311-99ea-4e34-bde6-870e8cfe6a2b\") "
Oct 30 21:54:56.125556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.125387    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:56.125556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.125401    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:56.125756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.125588    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:56.125756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.125613    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:56.125756 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.125686    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:56.127814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.127787    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:56.128030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.128008    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:56.128030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.128015    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "kube-api-access-ws9r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:56.128279 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.128258    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo" pod "a5b54311-99ea-4e34-bde6-870e8cfe6a2b" (UID: "a5b54311-99ea-4e34-bde6-870e8cfe6a2b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:56.226856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226819    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.226856 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226857    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226878    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ws9r4\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-kube-api-access-ws9r4\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226893    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226908    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226923    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226938    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226952    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.227070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.226965    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5b54311-99ea-4e34-bde6-870e8cfe6a2b-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:56.379070 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.379038    6738 generic.go:356] "Generic (PLEG): container finished" podID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerID="2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705" exitCode=0
Oct 30 21:54:56.379252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.379080    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"
Oct 30 21:54:56.379252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.379130    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" event={"ID":"a5b54311-99ea-4e34-bde6-870e8cfe6a2b","Type":"ContainerDied","Data":"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"}
Oct 30 21:54:56.379252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.379190    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8" event={"ID":"a5b54311-99ea-4e34-bde6-870e8cfe6a2b","Type":"ContainerDied","Data":"24cfeeaa02d0424563728c3c103e8a632717e00747aa313c1a3125ed9dc5d596"}
Oct 30 21:54:56.379252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.379219    6738 scope.go:122] "RemoveContainer" containerID="2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"
Oct 30 21:54:56.391216 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.391201    6738 scope.go:122] "RemoveContainer" containerID="2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"
Oct 30 21:54:56.391564 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:56.391542    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705\": container with ID starting with 2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705 not found: ID does not exist" containerID="2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"
Oct 30 21:54:56.391680 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.391573    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705"} err="failed to get container status \"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705\": rpc error: code = NotFound desc = could not find container \"2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705\": container with ID starting with 2ab61214409dd5d8b555d076da3c539be1de6f324654b7605267e039481c0705 not found: ID does not exist"
Oct 30 21:54:56.406853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.406831    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:54:56.412576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.412552    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-a378a2-openshift-default-5b788d5d5-f6ng8"]
Oct 30 21:54:56.606859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.606825    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:54:56.680688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.680606    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:56.865618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:56.865581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:57.172499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.170098    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" path="/var/lib/kubelet/pods/a5b54311-99ea-4e34-bde6-870e8cfe6a2b/volumes"
Oct 30 21:54:57.674928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.673772    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:57.791824 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.791527    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:54:57.828981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.828955    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:54:57.926882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.926855    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:57.950823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.950788    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.950987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.950831    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.950987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.950856    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.950987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.950927    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.950987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.950956    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.951213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951141    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951278 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951205    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.951334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951272    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.951334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951283    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951428    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951535    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.951625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951569    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951682 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951641    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951675    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.951764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951739    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.951873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951787    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.951873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951819    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.951873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951851    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.952031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951878    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.952031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951886    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.952031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951883    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.952031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.951949    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\") pod \"b76a6d50-5d68-437f-87de-34a4edc6ce98\" (UID: \"b76a6d50-5d68-437f-87de-34a4edc6ce98\") "
Oct 30 21:54:57.952031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.952019    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\") pod \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\" (UID: \"6b558560-ef9f-4ebc-b90c-3c350fc1bd35\") "
Oct 30 21:54:57.953206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.952852    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.953206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.952895    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.953206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.952970    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953322    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953417    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953462    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953479    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953494    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953507    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953522    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953536    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.953545 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.953549    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:57.954422 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.954352    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:54:57.955161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.954550    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:57.955161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.954897    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "kube-api-access-xx9wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:57.955397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.955373    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "kube-api-access-ntdrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:57.955617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.955531    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:54:57.956001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.955973    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:57.956001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.955982    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy" pod "6b558560-ef9f-4ebc-b90c-3c350fc1bd35" (UID: "6b558560-ef9f-4ebc-b90c-3c350fc1bd35"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.956175 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.956158    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:54:57.956269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:57.956254    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo" pod "b76a6d50-5d68-437f-87de-34a4edc6ce98" (UID: "b76a6d50-5d68-437f-87de-34a4edc6ce98"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055063    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055090    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055100    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx9wt\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-kube-api-access-xx9wt\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055109    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b76a6d50-5d68-437f-87de-34a4edc6ce98-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055118    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055126    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntdrs\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-kube-api-access-ntdrs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055137    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055145    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b558560-ef9f-4ebc-b90c-3c350fc1bd35-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055149 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055152    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.055821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.055162    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b76a6d50-5d68-437f-87de-34a4edc6ce98-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:54:58.396192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.396156    6738 generic.go:356] "Generic (PLEG): container finished" podID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerID="c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719" exitCode=0
Oct 30 21:54:58.396396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.396222    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"
Oct 30 21:54:58.396396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.396282    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" event={"ID":"b76a6d50-5d68-437f-87de-34a4edc6ce98","Type":"ContainerDied","Data":"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"}
Oct 30 21:54:58.396396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.396364    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" event={"ID":"b76a6d50-5d68-437f-87de-34a4edc6ce98","Type":"ContainerDied","Data":"1d45fd02a778009c06c8d092c67f4413804d02b9aa60e1284e78bd3dbd46c7ce"}
Oct 30 21:54:58.396396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.396391    6738 scope.go:122] "RemoveContainer" containerID="c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"
Oct 30 21:54:58.400732 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.400701    6738 generic.go:356] "Generic (PLEG): container finished" podID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerID="d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d" exitCode=0
Oct 30 21:54:58.400889 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.400787    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"
Oct 30 21:54:58.401104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.401077    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" event={"ID":"6b558560-ef9f-4ebc-b90c-3c350fc1bd35","Type":"ContainerDied","Data":"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"}
Oct 30 21:54:58.401271 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.401254    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt" event={"ID":"6b558560-ef9f-4ebc-b90c-3c350fc1bd35","Type":"ContainerDied","Data":"0fd7ebb7820c66bb6052b08d03e1f33ea2161dcdda42e5e288886e6942c3f128"}
Oct 30 21:54:58.410725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.410705    6738 scope.go:122] "RemoveContainer" containerID="c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"
Oct 30 21:54:58.411046 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:58.411024    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719\": container with ID starting with c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719 not found: ID does not exist" containerID="c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"
Oct 30 21:54:58.411125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.411055    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719"} err="failed to get container status \"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719\": rpc error: code = NotFound desc = could not find container \"c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719\": container with ID starting with c0bc6a26a764eb10e27e67fba9c4655a66beb9432f4f43b3c469bd966dcd6719 not found: ID does not exist"
Oct 30 21:54:58.411125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.411073    6738 scope.go:122] "RemoveContainer" containerID="d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"
Oct 30 21:54:58.423008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.422990    6738 scope.go:122] "RemoveContainer" containerID="d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"
Oct 30 21:54:58.423295 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:54:58.423279    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d\": container with ID starting with d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d not found: ID does not exist" containerID="d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"
Oct 30 21:54:58.423349 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.423300    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d"} err="failed to get container status \"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d\": rpc error: code = NotFound desc = could not find container \"d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d\": container with ID starting with d193d7972908d0da58936d031cadc504b34addd3c2d108dcbfc97e37cff42e7d not found: ID does not exist"
Oct 30 21:54:58.434700 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.434679    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:54:58.438116 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.438093    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9"]
Oct 30 21:54:58.450413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.450391    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:54:58.457230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.457202    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-4t2lt"]
Oct 30 21:54:58.677293 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.677214    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:54:58.724195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.724151    6738 patch_prober.go:29] interesting pod/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 21:54:58.724364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:58.724217    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-extmod-a-fb2b58-openshift-default-66696d54d9-5shx9" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 21:54:59.143740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:59.143680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:54:59.202096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:59.199884    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" path="/var/lib/kubelet/pods/6b558560-ef9f-4ebc-b90c-3c350fc1bd35/volumes"
Oct 30 21:54:59.202096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:59.200696    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" path="/var/lib/kubelet/pods/b76a6d50-5d68-437f-87de-34a4edc6ce98/volumes"
Oct 30 21:54:59.858556 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:54:59.858529    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:00.627420 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:00.624958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:01.374835 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:01.374798    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:01.503354 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:01.503319    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:01.766063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:01.765972    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:55:01.768086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:01.768053    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" containerID="cri-o://4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02" gracePeriod=30
Oct 30 21:55:02.265620 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.265599    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:02.439757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.439596    6738 generic.go:356] "Generic (PLEG): container finished" podID="74e13002-b907-4fad-ae17-2ff178c1550c" containerID="7879c0c320bee35614ce816b22c97b6d8f7b88c080bb66fa81f7b44398c22d6e" exitCode=0
Oct 30 21:55:02.440328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.440282    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" event={"ID":"74e13002-b907-4fad-ae17-2ff178c1550c","Type":"ContainerDied","Data":"7879c0c320bee35614ce816b22c97b6d8f7b88c080bb66fa81f7b44398c22d6e"}
Oct 30 21:55:02.442502 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.442450    6738 generic.go:356] "Generic (PLEG): container finished" podID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerID="3d2424709325858da11d145b6034c6398785639a5af4051e5816f63b980e5531" exitCode=0
Oct 30 21:55:02.442609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.442561    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" event={"ID":"428bc8c4-362d-43af-91ed-69c6e01ee773","Type":"ContainerDied","Data":"3d2424709325858da11d145b6034c6398785639a5af4051e5816f63b980e5531"}
Oct 30 21:55:02.512032 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.512006    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:55:02.547626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.547595    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\" (UniqueName: \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\") pod \"428bc8c4-362d-43af-91ed-69c6e01ee773\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") "
Oct 30 21:55:02.548101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.548080    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\") pod \"428bc8c4-362d-43af-91ed-69c6e01ee773\" (UID: \"428bc8c4-362d-43af-91ed-69c6e01ee773\") "
Oct 30 21:55:02.548623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.548580    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume" pod "428bc8c4-362d-43af-91ed-69c6e01ee773" (UID: "428bc8c4-362d-43af-91ed-69c6e01ee773"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:02.549493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.549198    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/428bc8c4-362d-43af-91ed-69c6e01ee773-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.552418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.552381    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7" pod "428bc8c4-362d-43af-91ed-69c6e01ee773" (UID: "428bc8c4-362d-43af-91ed-69c6e01ee773"). InnerVolumeSpecName "kube-api-access-r7fq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:02.570893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.570871    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:55:02.649907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.649875    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\" (UniqueName: \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\") pod \"74e13002-b907-4fad-ae17-2ff178c1550c\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") "
Oct 30 21:55:02.650092 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.650080    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\") pod \"74e13002-b907-4fad-ae17-2ff178c1550c\" (UID: \"74e13002-b907-4fad-ae17-2ff178c1550c\") "
Oct 30 21:55:02.650476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.650421    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume" pod "74e13002-b907-4fad-ae17-2ff178c1550c" (UID: "74e13002-b907-4fad-ae17-2ff178c1550c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:02.650476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.650457    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7fq7\" (UniqueName: \"kubernetes.io/projected/428bc8c4-362d-43af-91ed-69c6e01ee773-kube-api-access-r7fq7\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.652172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.652146    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2" pod "74e13002-b907-4fad-ae17-2ff178c1550c" (UID: "74e13002-b907-4fad-ae17-2ff178c1550c"). InnerVolumeSpecName "kube-api-access-bxdr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:02.751980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.751893    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74e13002-b907-4fad-ae17-2ff178c1550c-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.751980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.751926    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxdr2\" (UniqueName: \"kubernetes.io/projected/74e13002-b907-4fad-ae17-2ff178c1550c-kube-api-access-bxdr2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:02.983007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:02.982969    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:03.453667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.453492    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj" event={"ID":"74e13002-b907-4fad-ae17-2ff178c1550c","Type":"ContainerDied","Data":"96118e2de92288b6366b83cc667ac94cd6b7f0ed63bb352213c97a67d5e2a47e"}
Oct 30 21:55:03.453667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.453524    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"
Oct 30 21:55:03.453667 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.453578    6738 scope.go:122] "RemoveContainer" containerID="7879c0c320bee35614ce816b22c97b6d8f7b88c080bb66fa81f7b44398c22d6e"
Oct 30 21:55:03.455183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.455155    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx" event={"ID":"428bc8c4-362d-43af-91ed-69c6e01ee773","Type":"ContainerDied","Data":"5761fcd4c186f5e5bc2860df9f1cbf1dfd2bc46538b832fd8a96e9db1a2ab1c2"}
Oct 30 21:55:03.455300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.455202    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"
Oct 30 21:55:03.464861 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.464828    6738 scope.go:122] "RemoveContainer" containerID="3d2424709325858da11d145b6034c6398785639a5af4051e5816f63b980e5531"
Oct 30 21:55:03.475770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.475751    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:55:03.480109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.480087    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-a378a2-df78cf9-k2rmj"]
Oct 30 21:55:03.493292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.493268    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:55:03.496924 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.496904    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-a378a2-5c5cf75495-xwxcx"]
Oct 30 21:55:03.775311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:03.775241    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.330102 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.330067    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:55:04.330396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.330367    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" podUID="235355db-f889-4889-a406-d3e371902f52" containerName="maas-api" containerID="cri-o://ff051bf63441af60195c9e813dcd8f71d087f749c681d0a239a97383882a2c97" gracePeriod=30
Oct 30 21:55:04.391521 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.390514    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.468901 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.468871    6738 generic.go:356] "Generic (PLEG): container finished" podID="235355db-f889-4889-a406-d3e371902f52" containerID="ff051bf63441af60195c9e813dcd8f71d087f749c681d0a239a97383882a2c97" exitCode=0
Oct 30 21:55:04.469630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.469361    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" event={"ID":"235355db-f889-4889-a406-d3e371902f52","Type":"ContainerDied","Data":"ff051bf63441af60195c9e813dcd8f71d087f749c681d0a239a97383882a2c97"}
Oct 30 21:55:04.604085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.603959    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.780551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.780467    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:04.861379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.861354    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:55:04.938028 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:04.937933    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.015775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.015745    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\" (UniqueName: \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\") pod \"235355db-f889-4889-a406-d3e371902f52\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") "
Oct 30 21:55:05.015948 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.015855    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\") pod \"235355db-f889-4889-a406-d3e371902f52\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") "
Oct 30 21:55:05.016036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.015973    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") pod \"235355db-f889-4889-a406-d3e371902f52\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") "
Oct 30 21:55:05.016036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.016006    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\") pod \"235355db-f889-4889-a406-d3e371902f52\" (UID: \"235355db-f889-4889-a406-d3e371902f52\") "
Oct 30 21:55:05.016292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.016265    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca" pod "235355db-f889-4889-a406-d3e371902f52" (UID: "235355db-f889-4889-a406-d3e371902f52"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:05.016601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.016576    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/235355db-f889-4889-a406-d3e371902f52-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:05.018348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.018324    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs" pod "235355db-f889-4889-a406-d3e371902f52" (UID: "235355db-f889-4889-a406-d3e371902f52"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:05.018571 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.018556    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls" pod "235355db-f889-4889-a406-d3e371902f52" (UID: "235355db-f889-4889-a406-d3e371902f52"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:05.019611 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.019585    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq" pod "235355db-f889-4889-a406-d3e371902f52" (UID: "235355db-f889-4889-a406-d3e371902f52"). InnerVolumeSpecName "kube-api-access-bt5sq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:05.117651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.117588    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:05.117651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.117612    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/235355db-f889-4889-a406-d3e371902f52-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:05.117651 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.117622    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bt5sq\" (UniqueName: \"kubernetes.io/projected/235355db-f889-4889-a406-d3e371902f52-kube-api-access-bt5sq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:05.164610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.164580    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" path="/var/lib/kubelet/pods/428bc8c4-362d-43af-91ed-69c6e01ee773/volumes"
Oct 30 21:55:05.165000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.164986    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" path="/var/lib/kubelet/pods/74e13002-b907-4fad-ae17-2ff178c1550c/volumes"
Oct 30 21:55:05.309274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.309243    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:05.480643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.480552    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c" event={"ID":"235355db-f889-4889-a406-d3e371902f52","Type":"ContainerDied","Data":"04120f5772b2b215aefa5ae8e1eb89e93e6e566d26aa3e2848425c50ffc8745b"}
Oct 30 21:55:05.480643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.480625    6738 scope.go:122] "RemoveContainer" containerID="ff051bf63441af60195c9e813dcd8f71d087f749c681d0a239a97383882a2c97"
Oct 30 21:55:05.481180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.480560    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"
Oct 30 21:55:05.502725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.502701    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:55:05.506841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:05.506818    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-f9e5a7-bd6fdfdb-bxg4c"]
Oct 30 21:55:06.046400 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:06.046367    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:06.596567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:06.587527    6738 patch_prober.go:29] interesting pod/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:55:06.596567 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:06.587589    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:55:06.758136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:06.758108    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:06.912283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:06.912212    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.160681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.160655    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:55:07.165248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.165223    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="235355db-f889-4889-a406-d3e371902f52" path="/var/lib/kubelet/pods/235355db-f889-4889-a406-d3e371902f52/volumes"
Oct 30 21:55:07.231740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.231684    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.245593 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245565    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.245724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245624    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.245724 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245712    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.245866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245848    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.245921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245885    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.245962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.245941    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.246041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.246027    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.246103 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.246089    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.246191 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.246177    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\") pod \"a3638232-08bf-420e-b13b-12cb6dce0fd7\" (UID: \"a3638232-08bf-420e-b13b-12cb6dce0fd7\") "
Oct 30 21:55:07.247491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.247466    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.248675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.248566    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:07.248675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.248607    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.248675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.248661    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.248892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.248873    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.250654 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.250607    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:07.251249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.251226    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:55:07.251343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.251293    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "kube-api-access-xps8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:07.251403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.251359    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy" pod "a3638232-08bf-420e-b13b-12cb6dce0fd7" (UID: "a3638232-08bf-420e-b13b-12cb6dce0fd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:07.348853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348808    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.348853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348849    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.348853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348860    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348870    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348880    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xps8w\" (UniqueName: \"kubernetes.io/projected/a3638232-08bf-420e-b13b-12cb6dce0fd7-kube-api-access-xps8w\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348888    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348896    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348903    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a3638232-08bf-420e-b13b-12cb6dce0fd7-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.349145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.348911    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a3638232-08bf-420e-b13b-12cb6dce0fd7-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:07.500089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.499988    6738 generic.go:356] "Generic (PLEG): container finished" podID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerID="4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02" exitCode=0
Oct 30 21:55:07.500089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.500042    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"
Oct 30 21:55:07.500089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.500080    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" event={"ID":"a3638232-08bf-420e-b13b-12cb6dce0fd7","Type":"ContainerDied","Data":"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"}
Oct 30 21:55:07.500387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.500160    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg" event={"ID":"a3638232-08bf-420e-b13b-12cb6dce0fd7","Type":"ContainerDied","Data":"348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96"}
Oct 30 21:55:07.500387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.500189    6738 scope.go:122] "RemoveContainer" containerID="4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"
Oct 30 21:55:07.527189 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.527166    6738 scope.go:122] "RemoveContainer" containerID="4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"
Oct 30 21:55:07.530656 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:07.530625    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02\": container with ID starting with 4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02 not found: ID does not exist" containerID="4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"
Oct 30 21:55:07.530766 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.530666    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02"} err="failed to get container status \"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02\": rpc error: code = NotFound desc = could not find container \"4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02\": container with ID starting with 4a84b31499adb4fbe37d00d49df2fa8ff269f41a9299c77c47fefbb426f5db02 not found: ID does not exist"
Oct 30 21:55:07.543273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.543251    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:55:07.550415 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.550390    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bfc50e-openshift-default-55967d48cc-lcszg"]
Oct 30 21:55:07.562375 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:07.562346    6738 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3638232_08bf_420e_b13b_12cb6dce0fd7.slice/crio-348e2bb53296034e6f67442657ffb10e1262273a88651742f63e32e05e3edf96\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3638232_08bf_420e_b13b_12cb6dce0fd7.slice\": RecentStats: unable to find data in memory cache]"
Oct 30 21:55:07.923341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.923301    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:07.980182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.980143    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:55:07.980663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:07.980631    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" podUID="7c7b1113-9813-4257-bb65-3b1a03ce1455" containerName="maas-api" containerID="cri-o://0e4ec613fe4c49731fc12029cb1c37e6ab77dcbf7f7815e557e7f67feede8afc" gracePeriod=30
Oct 30 21:55:08.078130 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.078085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:08.521879 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.521840    6738 generic.go:356] "Generic (PLEG): container finished" podID="7c7b1113-9813-4257-bb65-3b1a03ce1455" containerID="0e4ec613fe4c49731fc12029cb1c37e6ab77dcbf7f7815e557e7f67feede8afc" exitCode=0
Oct 30 21:55:08.522016 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.521992    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" event={"ID":"7c7b1113-9813-4257-bb65-3b1a03ce1455","Type":"ContainerDied","Data":"0e4ec613fe4c49731fc12029cb1c37e6ab77dcbf7f7815e557e7f67feede8afc"}
Oct 30 21:55:08.522094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.522043    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7" event={"ID":"7c7b1113-9813-4257-bb65-3b1a03ce1455","Type":"ContainerDied","Data":"e760cdcbd4c94c2293fa20e03d0c7a6a8b66de00a78bfca9c32925b57f799511"}
Oct 30 21:55:08.522094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.522065    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e760cdcbd4c94c2293fa20e03d0c7a6a8b66de00a78bfca9c32925b57f799511"
Oct 30 21:55:08.534875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.534847    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:55:08.618241 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.618200    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\") pod \"7c7b1113-9813-4257-bb65-3b1a03ce1455\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") "
Oct 30 21:55:08.618757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.618424    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") pod \"7c7b1113-9813-4257-bb65-3b1a03ce1455\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") "
Oct 30 21:55:08.618757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.618505    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\") pod \"7c7b1113-9813-4257-bb65-3b1a03ce1455\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") "
Oct 30 21:55:08.618757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.618546    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\") pod \"7c7b1113-9813-4257-bb65-3b1a03ce1455\" (UID: \"7c7b1113-9813-4257-bb65-3b1a03ce1455\") "
Oct 30 21:55:08.619162 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.619126    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca" pod "7c7b1113-9813-4257-bb65-3b1a03ce1455" (UID: "7c7b1113-9813-4257-bb65-3b1a03ce1455"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:08.620841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.620726    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs" pod "7c7b1113-9813-4257-bb65-3b1a03ce1455" (UID: "7c7b1113-9813-4257-bb65-3b1a03ce1455"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:08.620956 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.620919    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql" pod "7c7b1113-9813-4257-bb65-3b1a03ce1455" (UID: "7c7b1113-9813-4257-bb65-3b1a03ce1455"). InnerVolumeSpecName "kube-api-access-5tgql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:08.624572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.624549    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls" pod "7c7b1113-9813-4257-bb65-3b1a03ce1455" (UID: "7c7b1113-9813-4257-bb65-3b1a03ce1455"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:08.682382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.682348    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:08.720219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.720183    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:08.720219 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.720223    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tgql\" (UniqueName: \"kubernetes.io/projected/7c7b1113-9813-4257-bb65-3b1a03ce1455-kube-api-access-5tgql\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:08.720499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.720239    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7c7b1113-9813-4257-bb65-3b1a03ce1455-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:08.720499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.720253    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c7b1113-9813-4257-bb65-3b1a03ce1455-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:08.804811 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:08.804730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:09.165265 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.165235    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" path="/var/lib/kubelet/pods/a3638232-08bf-420e-b13b-12cb6dce0fd7/volumes"
Oct 30 21:55:09.228509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.228478    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:09.532234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.532149    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"
Oct 30 21:55:09.551943 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.551308    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:55:09.554536 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.554514    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-15e79c-67459bb7f5-r9rk7"]
Oct 30 21:55:09.734690 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:09.734662    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:10.525460 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:10.524679    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:11.183600 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:11.183562    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c7b1113-9813-4257-bb65-3b1a03ce1455" path="/var/lib/kubelet/pods/7c7b1113-9813-4257-bb65-3b1a03ce1455/volumes"
Oct 30 21:55:11.647873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:11.647654    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" podUID="4b1a1130-125c-4344-8efc-00466623cf8b" containerName="main" containerID="cri-o://df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c" gracePeriod=45
Oct 30 21:55:11.944560 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:11.944531    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:55:12.041509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.032468    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:12.071158 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071126    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071177    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071219    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071251    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071286    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\" (UniqueName: \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071353    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\" (UniqueName: \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\") pod \"4b1a1130-125c-4344-8efc-00466623cf8b\" (UID: \"4b1a1130-125c-4344-8efc-00466623cf8b\") "
Oct 30 21:55:12.071608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071412    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:12.071608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071512    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:12.071608 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071529    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:12.072026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.071985    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.072026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.072015    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.072026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.072030    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.074094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.073994    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:12.074094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.074067    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:12.074094 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.074081    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c" pod "4b1a1130-125c-4344-8efc-00466623cf8b" (UID: "4b1a1130-125c-4344-8efc-00466623cf8b"). InnerVolumeSpecName "kube-api-access-x5l5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:12.173046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.172992    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4b1a1130-125c-4344-8efc-00466623cf8b-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.173046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.173014    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4b1a1130-125c-4344-8efc-00466623cf8b-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.173046 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.173023    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5l5c\" (UniqueName: \"kubernetes.io/projected/4b1a1130-125c-4344-8efc-00466623cf8b-kube-api-access-x5l5c\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:12.562695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.562608    6738 generic.go:356] "Generic (PLEG): container finished" podID="4b1a1130-125c-4344-8efc-00466623cf8b" containerID="df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c" exitCode=0
Oct 30 21:55:12.562695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.562661    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"
Oct 30 21:55:12.562921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.562698    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" event={"ID":"4b1a1130-125c-4344-8efc-00466623cf8b","Type":"ContainerDied","Data":"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"}
Oct 30 21:55:12.562921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.562741    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk" event={"ID":"4b1a1130-125c-4344-8efc-00466623cf8b","Type":"ContainerDied","Data":"922139d3607904ed9608b5958e1ee9ba37cd43da5dbe659efc32d1fadd18b6bc"}
Oct 30 21:55:12.562921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.562759    6738 scope.go:122] "RemoveContainer" containerID="df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"
Oct 30 21:55:12.573939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.573921    6738 scope.go:122] "RemoveContainer" containerID="df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"
Oct 30 21:55:12.574249 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:12.574224    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c\": container with ID starting with df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c not found: ID does not exist" containerID="df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"
Oct 30 21:55:12.574299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.574259    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c"} err="failed to get container status \"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c\": rpc error: code = NotFound desc = could not find container \"df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c\": container with ID starting with df3c53ba95e2b57fa4878aea10c01c33725b31f7089268da4ef877c1f799ae8c not found: ID does not exist"
Oct 30 21:55:12.591353 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.591326    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:55:12.597882 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.597848    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-adf308/test-model-adf308-kserve-8678df9559-m86nk"]
Oct 30 21:55:12.922794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:12.922752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:13.164518 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:13.164477    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4b1a1130-125c-4344-8efc-00466623cf8b" path="/var/lib/kubelet/pods/4b1a1130-125c-4344-8efc-00466623cf8b/volumes"
Oct 30 21:55:13.745145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:13.745104    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.609394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:14.609361    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:14.787409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:14.787382    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:15.034058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:15.034023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:16.139640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:16.139522    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:17.242797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:17.242754    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:17.928154 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:17.928066    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.156886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:18.156823    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.365868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:18.365831    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.523190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:18.523101    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:18.742176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:18.742096    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:19.354980 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:19.354945    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:20.170523 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:20.170417    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.001542 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:20.999306    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.203808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:21.203770    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:21.829171 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:21.829133    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:22.512562 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:22.512501    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:55:22.676705 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:22.676664    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:22.871457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:22.867273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:23.062586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:23.062557    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:23.214101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:23.214022    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:23.635499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:23.635463    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:24.186629 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:24.186597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:24.983321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:24.983295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.727973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:25.727940    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:25.863902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:25.863867    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:26.268578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:26.268524    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:26.430221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:26.430186    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:26.700878 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:26.700842    6738 scope.go:122] "RemoveContainer" containerID="0e4ec613fe4c49731fc12029cb1c37e6ab77dcbf7f7815e557e7f67feede8afc"
Oct 30 21:55:27.080510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:27.080402    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:27.254001 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:27.253963    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:27.374351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:27.374326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:27.817995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:27.817917    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:28.333160 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:28.333130    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.063182 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.063144    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:55:29.077205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.077167    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:55:29.173781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.173753    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.322397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.322316    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:29.776274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.776243    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:55:29.776514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.776488    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" podUID="670ad8df-c084-4e8a-bb8f-6de1017cb331" containerName="maas-api" containerID="cri-o://fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb" gracePeriod=30
Oct 30 21:55:29.847904 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:29.847859    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:30.164784 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.164731    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:30.226487 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.222586    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:55:30.270639 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.270609    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\" (UniqueName: \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\") pod \"670ad8df-c084-4e8a-bb8f-6de1017cb331\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") "
Oct 30 21:55:30.270829 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.270703    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\") pod \"670ad8df-c084-4e8a-bb8f-6de1017cb331\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") "
Oct 30 21:55:30.270912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.270840    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\") pod \"670ad8df-c084-4e8a-bb8f-6de1017cb331\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") "
Oct 30 21:55:30.270912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.270880    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\") pod \"670ad8df-c084-4e8a-bb8f-6de1017cb331\" (UID: \"670ad8df-c084-4e8a-bb8f-6de1017cb331\") "
Oct 30 21:55:30.271522 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.271496    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca" pod "670ad8df-c084-4e8a-bb8f-6de1017cb331" (UID: "670ad8df-c084-4e8a-bb8f-6de1017cb331"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:30.276204 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.276180    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls" pod "670ad8df-c084-4e8a-bb8f-6de1017cb331" (UID: "670ad8df-c084-4e8a-bb8f-6de1017cb331"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:30.276590 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.276569    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs" pod "670ad8df-c084-4e8a-bb8f-6de1017cb331" (UID: "670ad8df-c084-4e8a-bb8f-6de1017cb331"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:30.277849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.277824    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d" pod "670ad8df-c084-4e8a-bb8f-6de1017cb331" (UID: "670ad8df-c084-4e8a-bb8f-6de1017cb331"). InnerVolumeSpecName "kube-api-access-ndb6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:30.317347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.317304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:30.373137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.373107    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/670ad8df-c084-4e8a-bb8f-6de1017cb331-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:30.373282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.373141    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:30.373282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.373156    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndb6d\" (UniqueName: \"kubernetes.io/projected/670ad8df-c084-4e8a-bb8f-6de1017cb331-kube-api-access-ndb6d\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:30.373282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.373170    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/670ad8df-c084-4e8a-bb8f-6de1017cb331-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:30.706292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.706258    6738 generic.go:356] "Generic (PLEG): container finished" podID="670ad8df-c084-4e8a-bb8f-6de1017cb331" containerID="fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb" exitCode=0
Oct 30 21:55:30.706509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.706302    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" event={"ID":"670ad8df-c084-4e8a-bb8f-6de1017cb331","Type":"ContainerDied","Data":"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"}
Oct 30 21:55:30.706509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.706360    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7" event={"ID":"670ad8df-c084-4e8a-bb8f-6de1017cb331","Type":"ContainerDied","Data":"1eed9f3e7eccff6d97fbd83018c205e7bc66b69927bf57d490968d0a944640ff"}
Oct 30 21:55:30.706509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.706382    6738 scope.go:122] "RemoveContainer" containerID="fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"
Oct 30 21:55:30.706509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.706316    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"
Oct 30 21:55:30.719525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.719502    6738 scope.go:122] "RemoveContainer" containerID="fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"
Oct 30 21:55:30.719821 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:30.719801    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb\": container with ID starting with fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb not found: ID does not exist" containerID="fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"
Oct 30 21:55:30.719902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.719831    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb"} err="failed to get container status \"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb\": rpc error: code = NotFound desc = could not find container \"fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb\": container with ID starting with fb695f19a0d3865fe91f3ece06a8d71501ebb301e6b584ac8244ddf1f9e3c5fb not found: ID does not exist"
Oct 30 21:55:30.733273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.733249    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:55:30.744660 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.744638    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-a9230f-54d6686cb7-jvsr7"]
Oct 30 21:55:30.786157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:30.786134    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:31.174483 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:31.174450    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="670ad8df-c084-4e8a-bb8f-6de1017cb331" path="/var/lib/kubelet/pods/670ad8df-c084-4e8a-bb8f-6de1017cb331/volumes"
Oct 30 21:55:31.219183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:31.219153    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:31.368673 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:31.368646    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:31.653358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:31.653325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:32.301844 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:32.301802    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.237647 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:33.237605    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.434008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:33.433939    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.623630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:33.623601    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:33.772942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:33.772914    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.063946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.063841    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerName="payload-processing" containerID="cri-o://b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9" gracePeriod=25
Oct 30 21:55:34.078448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.078407    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerName="payload-pre-processing" containerID="cri-o://9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b" gracePeriod=25
Oct 30 21:55:34.111213 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.111181    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.368997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.368975    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:55:34.398471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.398449    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:55:34.444799 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.444774    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\" (UniqueName: \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\") pod \"f3126c2f-abf7-414c-904a-cc39332c3e84\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") "
Oct 30 21:55:34.445276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.444807    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\" (UniqueName: \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\") pod \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") "
Oct 30 21:55:34.445276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.444845    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\") pod \"f3126c2f-abf7-414c-904a-cc39332c3e84\" (UID: \"f3126c2f-abf7-414c-904a-cc39332c3e84\") "
Oct 30 21:55:34.445276 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.444866    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\") pod \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\" (UID: \"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de\") "
Oct 30 21:55:34.445825 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.445787    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume" pod "f3126c2f-abf7-414c-904a-cc39332c3e84" (UID: "f3126c2f-abf7-414c-904a-cc39332c3e84"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:34.446042 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.446019    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume" pod "e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" (UID: "e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:34.447900 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.447849    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x" pod "e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" (UID: "e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de"). InnerVolumeSpecName "kube-api-access-cfq6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:34.453107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.453069    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96" pod "f3126c2f-abf7-414c-904a-cc39332c3e84" (UID: "f3126c2f-abf7-414c-904a-cc39332c3e84"). InnerVolumeSpecName "kube-api-access-c7n96". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:34.546928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.546898    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7n96\" (UniqueName: \"kubernetes.io/projected/f3126c2f-abf7-414c-904a-cc39332c3e84-kube-api-access-c7n96\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:34.546928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.546925    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfq6x\" (UniqueName: \"kubernetes.io/projected/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-kube-api-access-cfq6x\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:34.546928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.546934    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3126c2f-abf7-414c-904a-cc39332c3e84-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:34.547247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.546943    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:34.628123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.628085    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:34.762124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.762089    6738 generic.go:356] "Generic (PLEG): container finished" podID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerID="9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b" exitCode=0
Oct 30 21:55:34.762316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.762150    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"
Oct 30 21:55:34.762316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.762213    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" event={"ID":"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de","Type":"ContainerDied","Data":"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"}
Oct 30 21:55:34.762316 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.762296    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" event={"ID":"e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de","Type":"ContainerDied","Data":"9f62ed21f3f8bd934edc8762d093021ded282272cb250f8e6cf3b13eed85e3e8"}
Oct 30 21:55:34.762550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.762346    6738 scope.go:122] "RemoveContainer" containerID="9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"
Oct 30 21:55:34.763963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.763929    6738 generic.go:356] "Generic (PLEG): container finished" podID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerID="b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9" exitCode=0
Oct 30 21:55:34.764020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.764009    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" event={"ID":"f3126c2f-abf7-414c-904a-cc39332c3e84","Type":"ContainerDied","Data":"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"}
Oct 30 21:55:34.764064 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.764040    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" event={"ID":"f3126c2f-abf7-414c-904a-cc39332c3e84","Type":"ContainerDied","Data":"65a14ff9cdafba3ea62d8e906fd733b2c7ce75cddb33e6692bc6249754ae1f4f"}
Oct 30 21:55:34.764101 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.764095    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"
Oct 30 21:55:34.774971 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.774945    6738 scope.go:122] "RemoveContainer" containerID="9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"
Oct 30 21:55:34.775262 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:34.775240    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b\": container with ID starting with 9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b not found: ID does not exist" containerID="9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"
Oct 30 21:55:34.775348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.775270    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b"} err="failed to get container status \"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b\": rpc error: code = NotFound desc = could not find container \"9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b\": container with ID starting with 9703cbda15853d3fe2c14f3d122094ec3ceb1914212fd4b29dbc079343f3c47b not found: ID does not exist"
Oct 30 21:55:34.775348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.775294    6738 scope.go:122] "RemoveContainer" containerID="b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"
Oct 30 21:55:34.785899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.785882    6738 scope.go:122] "RemoveContainer" containerID="b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"
Oct 30 21:55:34.786141 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:34.786125    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9\": container with ID starting with b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9 not found: ID does not exist" containerID="b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"
Oct 30 21:55:34.786186 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.786148    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9"} err="failed to get container status \"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9\": rpc error: code = NotFound desc = could not find container \"b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9\": container with ID starting with b21eae9f14d8e9df9a0e57e06bf35e5fb5ace46d619badc3e63e4ca1fac042b9 not found: ID does not exist"
Oct 30 21:55:34.796234 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.796210    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:55:34.801966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.801945    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt"]
Oct 30 21:55:34.812413 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.812390    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:55:34.818180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.818161    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw"]
Oct 30 21:55:34.884341 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:34.884273    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.030635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.030596    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.196527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.196231    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" path="/var/lib/kubelet/pods/e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de/volumes"
Oct 30 21:55:35.197466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.197210    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" path="/var/lib/kubelet/pods/f3126c2f-abf7-414c-904a-cc39332c3e84/volumes"
Oct 30 21:55:35.199716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.197839    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:55:35.199716 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.198098    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy" containerID="cri-o://692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48" gracePeriod=30
Oct 30 21:55:35.218921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.218894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:35.337622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.337570    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-a9230f-df7b8cd6c-7xzbw" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.90:50052\" within 1s: context deadline exceeded"
Oct 30 21:55:35.338205 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.338173    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-a9230f-5dcc6f948c-jxbxt" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerName="payload-pre-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.89:50052\" within 1s: context deadline exceeded"
Oct 30 21:55:35.338526 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:55:35.338513    6738 logging.go:55] [core] [Channel #5907 SubChannel #5909]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.90:50052", ServerName: "10.134.0.90:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.90:50052: operation was canceled"
Oct 30 21:55:35.338797 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:55:35.338783    6738 logging.go:55] [core] [Channel #5906 SubChannel #5908]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.89:50052", ServerName: "10.134.0.89:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.89:50052: operation was canceled"
Oct 30 21:55:35.620395 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:35.620303    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:36.699462 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:36.699261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:37.432695 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:37.432666    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:37.627452 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:37.627408    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:37.766138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:37.766051    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:37.959009 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:37.958975    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:55:37.959292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:37.959220    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" containerID="cri-o://125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7" gracePeriod=30
Oct 30 21:55:38.004210 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:38.004177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.284815 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:38.284787    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:38.459886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:38.459856    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:39.387252 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:39.387160    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.085531 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.085497    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.397122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.390496    6738 patch_prober.go:29] interesting pod/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:55:40.397122 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.390550    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:55:40.486135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.486113    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:55:40.564172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564140    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564248    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564423 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564392    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564430    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564493    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564530    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564554    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564578    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564625 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564601    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\") pod \"76714185-024e-496e-b444-f9d187dedf65\" (UID: \"76714185-024e-496e-b444-f9d187dedf65\") "
Oct 30 21:55:40.564850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564523    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:40.564850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564718    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:40.565214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.564968    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:40.565214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.565080    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:40.565214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.565176    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:40.565468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.565278    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.565468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.565301    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.565468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.565314    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.567207 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.567175    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:40.567340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.567217    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:40.567493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.567476    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "kube-api-access-z4jb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:40.567773 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.567747    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo" pod "76714185-024e-496e-b444-f9d187dedf65" (UID: "76714185-024e-496e-b444-f9d187dedf65"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666332    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4jb4\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-kube-api-access-z4jb4\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666367    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76714185-024e-496e-b444-f9d187dedf65-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666378    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76714185-024e-496e-b444-f9d187dedf65-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666387    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666396    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76714185-024e-496e-b444-f9d187dedf65-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.666409 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.666406    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76714185-024e-496e-b444-f9d187dedf65-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:40.694862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.694818    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.808995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.808941    6738 generic.go:356] "Generic (PLEG): container finished" podID="76714185-024e-496e-b444-f9d187dedf65" containerID="692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48" exitCode=0
Oct 30 21:55:40.809190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.809043    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"
Oct 30 21:55:40.809190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.809047    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerDied","Data":"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"}
Oct 30 21:55:40.809190 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.809166    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl" event={"ID":"76714185-024e-496e-b444-f9d187dedf65","Type":"ContainerDied","Data":"5584d15615bb7fa2c8fe36712cd5d22315074c14506ecb269013ad3d3a311acd"}
Oct 30 21:55:40.809351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.809193    6738 scope.go:122] "RemoveContainer" containerID="692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"
Oct 30 21:55:40.815770 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.815682    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:40.822533 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.822420    6738 scope.go:122] "RemoveContainer" containerID="662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"
Oct 30 21:55:40.836743 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.836714    6738 scope.go:122] "RemoveContainer" containerID="692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"
Oct 30 21:55:40.837076 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:40.837049    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48\": container with ID starting with 692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48 not found: ID does not exist" containerID="692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"
Oct 30 21:55:40.837185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.837087    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48"} err="failed to get container status \"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48\": rpc error: code = NotFound desc = could not find container \"692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48\": container with ID starting with 692471aab55027b4bc2a3fbf3e320e88264518478cd9ea83203e669943e80e48 not found: ID does not exist"
Oct 30 21:55:40.837185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.837106    6738 scope.go:122] "RemoveContainer" containerID="662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"
Oct 30 21:55:40.837410 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:40.837392    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf\": container with ID starting with 662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf not found: ID does not exist" containerID="662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"
Oct 30 21:55:40.837497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.837418    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf"} err="failed to get container status \"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf\": rpc error: code = NotFound desc = could not find container \"662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf\": container with ID starting with 662745ff831f86698e6a669a647fa59fe6f51c865e7b1c49a146220678cb8fdf not found: ID does not exist"
Oct 30 21:55:40.838005 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.837987    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:55:40.843836 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:40.843815    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-adf308-openshift-default-7d8768b5cf-8vrtl"]
Oct 30 21:55:41.142656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:41.142616    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:41.164907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:41.164873    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76714185-024e-496e-b444-f9d187dedf65" path="/var/lib/kubelet/pods/76714185-024e-496e-b444-f9d187dedf65/volumes"
Oct 30 21:55:41.750653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:41.750614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:42.427718 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:42.427677    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:42.595982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:42.595949    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:42.742013 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:42.741925    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.247614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.247587    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:55:43.403944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.403899    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:55:43.411538 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.411504    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.411875    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.411936    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412015    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412323    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412482    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412487    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412650    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412685    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412720    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:43.412778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412722    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412809    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.412890    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\") pod \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\" (UID: \"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4\") "
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413198    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413358    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413414    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413454    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413468    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.413489 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.413481    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.415656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.415625    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "kube-api-access-hfmss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:43.416875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.415968    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:55:43.416875 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.416180    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:43.418666 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.417856    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token" pod "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" (UID: "bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:43.432280 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.432251    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:55:43.514541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.514490    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.514541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.514525    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.514541 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.514540    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.514841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.514553    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.514841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.514568    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfmss\" (UniqueName: \"kubernetes.io/projected/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4-kube-api-access-hfmss\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:43.668723 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.668639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:43.832317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.832279    6738 generic.go:356] "Generic (PLEG): container finished" podID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerID="125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7" exitCode=0
Oct 30 21:55:43.832510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.832340    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"
Oct 30 21:55:43.832510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.832372    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" event={"ID":"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4","Type":"ContainerDied","Data":"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"}
Oct 30 21:55:43.832510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.832447    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr" event={"ID":"bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4","Type":"ContainerDied","Data":"e26ec5d6f38aa2aa95f910e71590c333d805f81fa99e523e225cb7c100b9c535"}
Oct 30 21:55:43.832510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.832460    6738 scope.go:122] "RemoveContainer" containerID="125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"
Oct 30 21:55:43.845099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.845079    6738 scope.go:122] "RemoveContainer" containerID="125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"
Oct 30 21:55:43.845425 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:43.845401    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7\": container with ID starting with 125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7 not found: ID does not exist" containerID="125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"
Oct 30 21:55:43.845515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.845470    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7"} err="failed to get container status \"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7\": rpc error: code = NotFound desc = could not find container \"125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7\": container with ID starting with 125f213146a081514519ae51e42f266f1960f8be052793f58fffb62de656c9e7 not found: ID does not exist"
Oct 30 21:55:43.866804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.866778    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:55:43.877063 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.877040    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-a9230f-openshift-default-767df789fc-2lcxr"]
Oct 30 21:55:43.913598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:43.913574    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:44.502823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:44.502791    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:45.165392 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:45.165358    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" path="/var/lib/kubelet/pods/bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4/volumes"
Oct 30 21:55:45.284081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:45.284046    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:45.866074 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:45.866042    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:55:45.969997 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:45.969962    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:46.087172 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:46.087128    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:46.225086 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:46.224994    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:47.122892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:47.122833    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:47.972255 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:47.972220    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:48.927602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:48.927565    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:49.180107 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:49.180027    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:50.036137 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:50.036103    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:50.867905 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:50.867862    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" podUID="e374033d-2102-4de2-8520-28eb00acdd15" containerName="payload-processing" containerID="cri-o://a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a" gracePeriod=25
Oct 30 21:55:51.004142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.004099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:51.176347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.176318    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:55:51.309503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.309469    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\") pod \"e374033d-2102-4de2-8520-28eb00acdd15\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") "
Oct 30 21:55:51.309503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.309509    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\" (UniqueName: \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\") pod \"e374033d-2102-4de2-8520-28eb00acdd15\" (UID: \"e374033d-2102-4de2-8520-28eb00acdd15\") "
Oct 30 21:55:51.309839 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.309818    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume" pod "e374033d-2102-4de2-8520-28eb00acdd15" (UID: "e374033d-2102-4de2-8520-28eb00acdd15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:51.310115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.310089    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e374033d-2102-4de2-8520-28eb00acdd15-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:51.311950 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.311922    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x" pod "e374033d-2102-4de2-8520-28eb00acdd15" (UID: "e374033d-2102-4de2-8520-28eb00acdd15"). InnerVolumeSpecName "kube-api-access-9dp2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:51.411010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.410937    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dp2x\" (UniqueName: \"kubernetes.io/projected/e374033d-2102-4de2-8520-28eb00acdd15-kube-api-access-9dp2x\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:51.763300 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.763173    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:51.899540 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.899506    6738 generic.go:356] "Generic (PLEG): container finished" podID="e374033d-2102-4de2-8520-28eb00acdd15" containerID="a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a" exitCode=0
Oct 30 21:55:51.900140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.899957    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"
Oct 30 21:55:51.900478 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.899981    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" event={"ID":"e374033d-2102-4de2-8520-28eb00acdd15","Type":"ContainerDied","Data":"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"}
Oct 30 21:55:51.900710 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.900689    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g" event={"ID":"e374033d-2102-4de2-8520-28eb00acdd15","Type":"ContainerDied","Data":"f3472e15367d9e3735080fae246f7980846180941400f732cc57cbb875d46347"}
Oct 30 21:55:51.901131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:51.901066    6738 scope.go:122] "RemoveContainer" containerID="a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"
Oct 30 21:55:52.050712 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.044700    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:52.065262 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.064817    6738 scope.go:122] "RemoveContainer" containerID="a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"
Oct 30 21:55:52.073876 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:52.073838    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a\": container with ID starting with a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a not found: ID does not exist" containerID="a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"
Oct 30 21:55:52.074041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.073882    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a"} err="failed to get container status \"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a\": rpc error: code = NotFound desc = could not find container \"a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a\": container with ID starting with a4ff111bfdd240765c1a1fb2b785b0a8ceffc6f5ff10721b85204fa7dfe4c72a not found: ID does not exist"
Oct 30 21:55:52.077721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.077691    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:55:52.086056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.086030    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-e8d315-768d7b6cbc-76c6g"]
Oct 30 21:55:52.263772 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.263715    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:52.519458 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:52.516760    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:55:52.581377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.581349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:52.716757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.716718    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:52.844711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:52.844634    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:53.163982 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:53.163950    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e374033d-2102-4de2-8520-28eb00acdd15" path="/var/lib/kubelet/pods/e374033d-2102-4de2-8520-28eb00acdd15/volumes"
Oct 30 21:55:53.185263 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:53.185240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:53.481655 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:53.481570    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:54.352954 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:54.352928    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:55.174120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:55.174086    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:55.944588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:55.944560    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:56.656658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:56.656628    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:55:56.657183 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:56.656895    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" podUID="baa40822-2e8f-4f91-8a17-3febe3a476fc" containerName="maas-api" containerID="cri-o://d1fa3ab5877ed28630fdb62706ce90e343d965e701256d3a160242a739558e4b" gracePeriod=30
Oct 30 21:55:56.751459 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:56.748479    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:57.002969 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.002859    6738 generic.go:356] "Generic (PLEG): container finished" podID="baa40822-2e8f-4f91-8a17-3febe3a476fc" containerID="d1fa3ab5877ed28630fdb62706ce90e343d965e701256d3a160242a739558e4b" exitCode=0
Oct 30 21:55:57.003377 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.003334    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" event={"ID":"baa40822-2e8f-4f91-8a17-3febe3a476fc","Type":"ContainerDied","Data":"d1fa3ab5877ed28630fdb62706ce90e343d965e701256d3a160242a739558e4b"}
Oct 30 21:55:57.063023 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.063001    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:55:57.113245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113214    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\") pod \"baa40822-2e8f-4f91-8a17-3febe3a476fc\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") "
Oct 30 21:55:57.113472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113301    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\" (UniqueName: \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\") pod \"baa40822-2e8f-4f91-8a17-3febe3a476fc\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") "
Oct 30 21:55:57.113472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113406    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\") pod \"baa40822-2e8f-4f91-8a17-3febe3a476fc\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") "
Oct 30 21:55:57.113472 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113453    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\") pod \"baa40822-2e8f-4f91-8a17-3febe3a476fc\" (UID: \"baa40822-2e8f-4f91-8a17-3febe3a476fc\") "
Oct 30 21:55:57.113827 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113799    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca" pod "baa40822-2e8f-4f91-8a17-3febe3a476fc" (UID: "baa40822-2e8f-4f91-8a17-3febe3a476fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:55:57.114014 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.113994    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baa40822-2e8f-4f91-8a17-3febe3a476fc-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:57.116309 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.116263    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls" pod "baa40822-2e8f-4f91-8a17-3febe3a476fc" (UID: "baa40822-2e8f-4f91-8a17-3febe3a476fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:57.116729 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.116698    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs" pod "baa40822-2e8f-4f91-8a17-3febe3a476fc" (UID: "baa40822-2e8f-4f91-8a17-3febe3a476fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:57.116956 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.116924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:57.117223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.117199    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm" pod "baa40822-2e8f-4f91-8a17-3febe3a476fc" (UID: "baa40822-2e8f-4f91-8a17-3febe3a476fc"). InnerVolumeSpecName "kube-api-access-9qgsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:57.215457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.215414    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:57.215457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.215459    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/baa40822-2e8f-4f91-8a17-3febe3a476fc-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:57.215630 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.215476    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qgsm\" (UniqueName: \"kubernetes.io/projected/baa40822-2e8f-4f91-8a17-3febe3a476fc-kube-api-access-9qgsm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:57.423249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.423214    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:57.704640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:57.704543    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:58.010907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.010834    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"
Oct 30 21:55:58.010907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.010875    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd" event={"ID":"baa40822-2e8f-4f91-8a17-3febe3a476fc","Type":"ContainerDied","Data":"52575a4986e4d30d185bc4a5aac4be03848df97b00e83ff83b4f9af2fd50b2dc"}
Oct 30 21:55:58.011143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.010931    6738 scope.go:122] "RemoveContainer" containerID="d1fa3ab5877ed28630fdb62706ce90e343d965e701256d3a160242a739558e4b"
Oct 30 21:55:58.036394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.036366    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:55:58.045411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.045386    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w1-27804d-5f89d5b957-6j4rd"]
Oct 30 21:55:58.380284 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.380252    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:58.432054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.432021    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" containerID="cri-o://28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301" gracePeriod=45
Oct 30 21:55:58.442197 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.442170    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" containerID="cri-o://08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0" gracePeriod=45
Oct 30 21:55:58.459858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.459829    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main" containerID="cri-o://acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690" gracePeriod=45
Oct 30 21:55:58.542664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.542635    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": dial tcp 10.134.0.53:8000: connect: connection refused"
Oct 30 21:55:58.832812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.832785    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:55:58.849863 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.849831    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:55:58.851962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.851942    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:55:58.964514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964392    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.964514 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964452    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.964764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964561    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.964764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964598    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.964764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964629    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.964927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964850    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.964927 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964885    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.964989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964967    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.964989 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.964974    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.965053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965002    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.965053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965037    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\" (UniqueName: \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.965144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965097    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.965144 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965134    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.965232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965157    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\" (UniqueName: \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.965232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965187    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\" (UniqueName: \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.965232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965218    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\" (UniqueName: \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.965403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965240    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.965403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965282    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\") pod \"753953a8-b9be-4e08-a259-6d990fde1708\" (UID: \"753953a8-b9be-4e08-a259-6d990fde1708\") "
Oct 30 21:55:58.965403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965312    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\") pod \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\" (UID: \"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc\") "
Oct 30 21:55:58.965403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965398    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\") pod \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\" (UID: \"dbf52a31-a847-4ee2-9fcd-9c0ecb274750\") "
Oct 30 21:55:58.965631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965476    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.965631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965502    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.965999    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966098    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966196    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966306    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966324    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966339    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966353    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966370    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966384    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966397    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966798    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.967128 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.966882    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.970403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.970376    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.970403 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.970397    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.970739 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.970699    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "kube-api-access-zqfq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.971379 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.971342    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "kube-api-access-pn7mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.971519 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.971421    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm" pod "753953a8-b9be-4e08-a259-6d990fde1708" (UID: "753953a8-b9be-4e08-a259-6d990fde1708"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.972336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.972297    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "kube-api-access-tmrhj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:55:58.972715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.972685    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs" pod "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" (UID: "5a8e6aa2-6089-43c4-a030-e0bef25f5ccc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:58.973120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.973085    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:55:58.974157 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:58.974138    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs" pod "dbf52a31-a847-4ee2-9fcd-9c0ecb274750" (UID: "dbf52a31-a847-4ee2-9fcd-9c0ecb274750"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:55:59.019453 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.019401    6738 generic.go:356] "Generic (PLEG): container finished" podID="753953a8-b9be-4e08-a259-6d990fde1708" containerID="acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690" exitCode=0
Oct 30 21:55:59.019588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.019493    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"
Oct 30 21:55:59.019588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.019494    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" event={"ID":"753953a8-b9be-4e08-a259-6d990fde1708","Type":"ContainerDied","Data":"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"}
Oct 30 21:55:59.019588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.019545    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v" event={"ID":"753953a8-b9be-4e08-a259-6d990fde1708","Type":"ContainerDied","Data":"8bc21c01383176dba3726887a29a28a523e8ba98736ce483f2de59140a62c837"}
Oct 30 21:55:59.019588 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.019568    6738 scope.go:122] "RemoveContainer" containerID="acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"
Oct 30 21:55:59.021539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.021221    6738 generic.go:356] "Generic (PLEG): container finished" podID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerID="28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301" exitCode=0
Oct 30 21:55:59.021539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.021320    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" event={"ID":"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc","Type":"ContainerDied","Data":"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"}
Oct 30 21:55:59.021539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.021348    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" event={"ID":"5a8e6aa2-6089-43c4-a030-e0bef25f5ccc","Type":"ContainerDied","Data":"f80179eb0965e52f48b8e45f921ee2948035deab0eaad15d4b51adf48dddde81"}
Oct 30 21:55:59.021539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.021404    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"
Oct 30 21:55:59.026486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.026347    6738 generic.go:356] "Generic (PLEG): container finished" podID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerID="08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0" exitCode=0
Oct 30 21:55:59.026486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.026401    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" event={"ID":"dbf52a31-a847-4ee2-9fcd-9c0ecb274750","Type":"ContainerDied","Data":"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"}
Oct 30 21:55:59.026486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.026427    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"
Oct 30 21:55:59.028104 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.026429    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs" event={"ID":"dbf52a31-a847-4ee2-9fcd-9c0ecb274750","Type":"ContainerDied","Data":"af8fa60e5935afe490d6d1efb22d815973158f89115221af1add0a89ff146fae"}
Oct 30 21:55:59.037566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.037083    6738 scope.go:122] "RemoveContainer" containerID="acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"
Oct 30 21:55:59.037727 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:59.037547    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690\": container with ID starting with acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690 not found: ID does not exist" containerID="acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"
Oct 30 21:55:59.037727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.037612    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690"} err="failed to get container status \"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690\": rpc error: code = NotFound desc = could not find container \"acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690\": container with ID starting with acde5853b79a5d229525b81611b399a40c13dd9d44228e56f11b01bc56e67690 not found: ID does not exist"
Oct 30 21:55:59.037727 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.037633    6738 scope.go:122] "RemoveContainer" containerID="28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"
Oct 30 21:55:59.047302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.047280    6738 scope.go:122] "RemoveContainer" containerID="28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"
Oct 30 21:55:59.047632 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:59.047608    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301\": container with ID starting with 28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301 not found: ID does not exist" containerID="28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"
Oct 30 21:55:59.047683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.047644    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301"} err="failed to get container status \"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301\": rpc error: code = NotFound desc = could not find container \"28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301\": container with ID starting with 28d814b6f89e7427c641d8a8b93036d73f1ec7d6791f323010a4b3adbb349301 not found: ID does not exist"
Oct 30 21:55:59.047683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.047666    6738 scope.go:122] "RemoveContainer" containerID="08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"
Oct 30 21:55:59.056702 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.056683    6738 scope.go:122] "RemoveContainer" containerID="08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"
Oct 30 21:55:59.057033 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:55:59.057010    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0\": container with ID starting with 08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0 not found: ID does not exist" containerID="08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"
Oct 30 21:55:59.057115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.057040    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0"} err="failed to get container status \"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0\": rpc error: code = NotFound desc = could not find container \"08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0\": container with ID starting with 08375a372b6ce0bc873b071e879230d72827ded11478d18adbd65a8833df3cb0 not found: ID does not exist"
Oct 30 21:55:59.063006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.062979    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:55:59.067994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.067961    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.067994 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.067995    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068010    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn7mm\" (UniqueName: \"kubernetes.io/projected/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-kube-api-access-pn7mm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068027    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068042    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068055    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmrhj\" (UniqueName: \"kubernetes.io/projected/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-kube-api-access-tmrhj\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068072    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqfq9\" (UniqueName: \"kubernetes.io/projected/753953a8-b9be-4e08-a259-6d990fde1708-kube-api-access-zqfq9\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068087    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/753953a8-b9be-4e08-a259-6d990fde1708-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068100    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbf52a31-a847-4ee2-9fcd-9c0ecb274750-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068112    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/753953a8-b9be-4e08-a259-6d990fde1708-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.068159 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.068125    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:55:59.074851 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.074827    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/premium-w3-bfc50e-kserve-659f8f4d86-lmr4v"]
Oct 30 21:55:59.089366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.089333    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:55:59.091380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.091359    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl"]
Oct 30 21:55:59.109681 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.109657    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:55:59.112448 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.112408    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bfc50e/distinct-w3-bfc50e-kserve-bbdf9777c-dc7gs"]
Oct 30 21:55:59.164800 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.164776    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" path="/var/lib/kubelet/pods/5a8e6aa2-6089-43c4-a030-e0bef25f5ccc/volumes"
Oct 30 21:55:59.165202 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.165187    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="753953a8-b9be-4e08-a259-6d990fde1708" path="/var/lib/kubelet/pods/753953a8-b9be-4e08-a259-6d990fde1708/volumes"
Oct 30 21:55:59.165562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.165549    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="baa40822-2e8f-4f91-8a17-3febe3a476fc" path="/var/lib/kubelet/pods/baa40822-2e8f-4f91-8a17-3febe3a476fc/volumes"
Oct 30 21:55:59.166198 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.166184    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" path="/var/lib/kubelet/pods/dbf52a31-a847-4ee2-9fcd-9c0ecb274750/volumes"
Oct 30 21:55:59.386479 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.384983    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:55:59.761534 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:55:59.761428    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bfc50e/distinct-2-w3-bfc50e-kserve-67b7bdf9d-qsjjl" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.56:8000/ready\": context deadline exceeded"
Oct 30 21:56:00.422126 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:00.422095    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:01.176776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:01.176730    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:01.366010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:01.365970    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:01.550220 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:01.550136    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:01.719832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:01.719801    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:02.127193 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:02.127155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:02.739932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:02.739894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:03.014895 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:03.014800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:03.332866 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:03.332787    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:04.238050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:04.238018    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:05.930931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:05.930900    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:07.097794 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:07.097761    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:56:07.359230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:07.359144    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:07.879763 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:07.879699    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:08.744713 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:08.744674    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:09.547350 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:09.547321    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:09.887185 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:09.887156    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.112962 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:10.112912    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.484471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:10.484424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:10.892826 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:10.892785    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:11.439366 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:11.439328    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:11.633888 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:11.633841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.028201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:12.028145    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.539180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:12.539153    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.737798 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:12.737759    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:12.909615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:12.909580    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:13.806510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:13.806472    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:56:13.946359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:13.946315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:14.192816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:14.192777    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:15.036946 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:15.036916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:15.317566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:15.317478    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:56:15.325964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:15.325934    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:56:15.361056 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:15.361031    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:56:15.510343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:15.510309    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:16.073637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:16.073600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:17.021621 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:17.021587    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:18.110517 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:18.110481    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:18.550623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:18.550578    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:18.816618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:18.816503    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" podUID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" containerName="payload-pre-processing" containerID="cri-o://1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94" gracePeriod=25
Oct 30 21:56:19.112367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.112198    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:19.122080 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.121777    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:56:19.175181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.174595    6738 generic.go:356] "Generic (PLEG): container finished" podID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" containerID="1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94" exitCode=0
Oct 30 21:56:19.175181 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.174718    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"
Oct 30 21:56:19.176253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.176032    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" event={"ID":"7d70d105-da83-4c2e-aab7-bca9dc599b5f","Type":"ContainerDied","Data":"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"}
Oct 30 21:56:19.176253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.176083    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7" event={"ID":"7d70d105-da83-4c2e-aab7-bca9dc599b5f","Type":"ContainerDied","Data":"31077d0ab30aedbe4b89f6207b45a86c99e9daa77054b41e04f2e00f076c735b"}
Oct 30 21:56:19.176253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.176114    6738 scope.go:122] "RemoveContainer" containerID="1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"
Oct 30 21:56:19.189561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.189537    6738 scope.go:122] "RemoveContainer" containerID="1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"
Oct 30 21:56:19.189896 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:19.189871    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94\": container with ID starting with 1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94 not found: ID does not exist" containerID="1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"
Oct 30 21:56:19.189976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.189909    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94"} err="failed to get container status \"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94\": rpc error: code = NotFound desc = could not find container \"1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94\": container with ID starting with 1345469c89dc103db481163d9816f513dd2fb79985dce83174fc93f31e98fa94 not found: ID does not exist"
Oct 30 21:56:19.267505 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.267480    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\") pod \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") "
Oct 30 21:56:19.267640 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.267627    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\" (UniqueName: \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\") pod \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\" (UID: \"7d70d105-da83-4c2e-aab7-bca9dc599b5f\") "
Oct 30 21:56:19.267921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.267884    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume" pod "7d70d105-da83-4c2e-aab7-bca9dc599b5f" (UID: "7d70d105-da83-4c2e-aab7-bca9dc599b5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:19.270030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.270012    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5" pod "7d70d105-da83-4c2e-aab7-bca9dc599b5f" (UID: "7d70d105-da83-4c2e-aab7-bca9dc599b5f"). InnerVolumeSpecName "kube-api-access-cnbg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:19.368503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.368476    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnbg5\" (UniqueName: \"kubernetes.io/projected/7d70d105-da83-4c2e-aab7-bca9dc599b5f-kube-api-access-cnbg5\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:19.368503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.368500    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d70d105-da83-4c2e-aab7-bca9dc599b5f-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:19.502674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.502643    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:56:19.505963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.505934    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-fb2b58-648d7fbc6c-rchk7"]
Oct 30 21:56:19.843884 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:19.843783    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:20.094355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:20.094261    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:20.520031 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:20.519993    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:20.771247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:20.771170    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:21.012749 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:21.012706    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:21.164711 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:21.164681    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" path="/var/lib/kubelet/pods/7d70d105-da83-4c2e-aab7-bca9dc599b5f/volumes"
Oct 30 21:56:21.286804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:21.285110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:21.528987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:21.528905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:22.079491 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.079461    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:22.121062 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.121031    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main" containerID="cri-o://cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b" gracePeriod=45
Oct 30 21:56:22.389416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.389385    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:56:22.518290 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:22.518257    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:56:22.552068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552039    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552092    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552193    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552212    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\" (UniqueName: \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552234    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\" (UniqueName: \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552263    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\") pod \"09aeb068-df15-48d0-9ec4-17a6558620a0\" (UID: \"09aeb068-df15-48d0-9ec4-17a6558620a0\") "
Oct 30 21:56:22.552637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552615    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:22.553025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552764    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:22.553025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.552834    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:22.556494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.555501    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:22.556494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.555601    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:22.556494 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.555750    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49" pod "09aeb068-df15-48d0-9ec4-17a6558620a0" (UID: "09aeb068-df15-48d0-9ec4-17a6558620a0"). InnerVolumeSpecName "kube-api-access-82f49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:22.653550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653482    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.653550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653512    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.653550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653521    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.653550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653530    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09aeb068-df15-48d0-9ec4-17a6558620a0-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.653550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653541    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82f49\" (UniqueName: \"kubernetes.io/projected/09aeb068-df15-48d0-9ec4-17a6558620a0-kube-api-access-82f49\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.653821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.653554    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09aeb068-df15-48d0-9ec4-17a6558620a0-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:22.964471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:22.964317    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:23.206429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.206387    6738 generic.go:356] "Generic (PLEG): container finished" podID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerID="cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b" exitCode=0
Oct 30 21:56:23.206628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.206482    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"
Oct 30 21:56:23.206628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.206513    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" event={"ID":"09aeb068-df15-48d0-9ec4-17a6558620a0","Type":"ContainerDied","Data":"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"}
Oct 30 21:56:23.206628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.206551    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4" event={"ID":"09aeb068-df15-48d0-9ec4-17a6558620a0","Type":"ContainerDied","Data":"a7b3f8f0d222bc1d3f197ce3b3281f46119a5dcdeaedf8bfec78207f5ab44812"}
Oct 30 21:56:23.206628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.206597    6738 scope.go:122] "RemoveContainer" containerID="cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"
Oct 30 21:56:23.221988 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.221966    6738 scope.go:122] "RemoveContainer" containerID="cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"
Oct 30 21:56:23.222388 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:23.222362    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b\": container with ID starting with cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b not found: ID does not exist" containerID="cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"
Oct 30 21:56:23.222551 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.222401    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b"} err="failed to get container status \"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b\": rpc error: code = NotFound desc = could not find container \"cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b\": container with ID starting with cc836fc95826ad0d940537e8982af81997c0593398b1df6fffa0b5a2121e776b not found: ID does not exist"
Oct 30 21:56:23.237803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.237779    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:56:23.249515 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.249487    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-3011c6/simulator-w4-3011c6-kserve-5985ffd9f5-dg8j4"]
Oct 30 21:56:23.301089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.301062    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:23.531352 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:23.531259    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:24.722113 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:24.722081    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:25.163894 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:25.163865    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" path="/var/lib/kubelet/pods/09aeb068-df15-48d0-9ec4-17a6558620a0/volumes"
Oct 30 21:56:25.754817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:25.754783    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:56:25.755320 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:25.755149    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" podUID="2a990500-0296-4a9f-8fd0-7cf50282841e" containerName="maas-api" containerID="cri-o://1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc" gracePeriod=30
Oct 30 21:56:25.859675 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:25.859637    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:26.123083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.123050    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:56:26.165643 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.165606    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:26.210596 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.210559    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\") pod \"2a990500-0296-4a9f-8fd0-7cf50282841e\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") "
Oct 30 21:56:26.210774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.210660    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\" (UniqueName: \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\") pod \"2a990500-0296-4a9f-8fd0-7cf50282841e\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") "
Oct 30 21:56:26.210774 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.210722    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\") pod \"2a990500-0296-4a9f-8fd0-7cf50282841e\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") "
Oct 30 21:56:26.210902 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.210773    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\") pod \"2a990500-0296-4a9f-8fd0-7cf50282841e\" (UID: \"2a990500-0296-4a9f-8fd0-7cf50282841e\") "
Oct 30 21:56:26.213614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.211629    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca" pod "2a990500-0296-4a9f-8fd0-7cf50282841e" (UID: "2a990500-0296-4a9f-8fd0-7cf50282841e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:26.214496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.214426    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls" pod "2a990500-0296-4a9f-8fd0-7cf50282841e" (UID: "2a990500-0296-4a9f-8fd0-7cf50282841e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:26.214757 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.214740    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2a990500-0296-4a9f-8fd0-7cf50282841e-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:26.214823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.214762    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:26.232192 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.232159    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs" pod "2a990500-0296-4a9f-8fd0-7cf50282841e" (UID: "2a990500-0296-4a9f-8fd0-7cf50282841e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:26.232310 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.232165    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m" pod "2a990500-0296-4a9f-8fd0-7cf50282841e" (UID: "2a990500-0296-4a9f-8fd0-7cf50282841e"). InnerVolumeSpecName "kube-api-access-h7z5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:26.237256 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.237230    6738 generic.go:356] "Generic (PLEG): container finished" podID="2a990500-0296-4a9f-8fd0-7cf50282841e" containerID="1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc" exitCode=0
Oct 30 21:56:26.237351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.237279    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" event={"ID":"2a990500-0296-4a9f-8fd0-7cf50282841e","Type":"ContainerDied","Data":"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"}
Oct 30 21:56:26.237351 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.237284    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"
Oct 30 21:56:26.237429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.237351    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8" event={"ID":"2a990500-0296-4a9f-8fd0-7cf50282841e","Type":"ContainerDied","Data":"331c9cb2debca2931140c3b6623e809f7e2b26c4de98be1ad4e0ac650a647648"}
Oct 30 21:56:26.237429 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.237374    6738 scope.go:122] "RemoveContainer" containerID="1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"
Oct 30 21:56:26.249270 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.249249    6738 scope.go:122] "RemoveContainer" containerID="1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"
Oct 30 21:56:26.249642 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:26.249621    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc\": container with ID starting with 1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc not found: ID does not exist" containerID="1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"
Oct 30 21:56:26.249777 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.249652    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc"} err="failed to get container status \"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc\": rpc error: code = NotFound desc = could not find container \"1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc\": container with ID starting with 1b12b344497717caf6ed882ef8ac635a4206c952df5d202b3d64a168999d9bdc not found: ID does not exist"
Oct 30 21:56:26.279742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.279686    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:56:26.282416 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.282389    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-3011c6-5bd6f4c955-n4zq8"]
Oct 30 21:56:26.316226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.316194    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7z5m\" (UniqueName: \"kubernetes.io/projected/2a990500-0296-4a9f-8fd0-7cf50282841e-kube-api-access-h7z5m\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:26.316226 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.316225    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2a990500-0296-4a9f-8fd0-7cf50282841e-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:26.368853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.368820    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:26.955612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:26.955579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:27.163679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:27.163651    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a990500-0296-4a9f-8fd0-7cf50282841e" path="/var/lib/kubelet/pods/2a990500-0296-4a9f-8fd0-7cf50282841e/volumes"
Oct 30 21:56:27.604215 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:27.604128    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:28.033269 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:28.033223    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:28.943951 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:28.943916    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:29.999456 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:29.995404    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:30.339717 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.339570    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" containerID="cri-o://2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7" gracePeriod=45
Oct 30 21:56:30.346115 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.346074    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main" containerID="cri-o://464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06" gracePeriod=45
Oct 30 21:56:30.385529 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.385497    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main" containerID="cri-o://81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d" gracePeriod=45
Oct 30 21:56:30.513407 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.513317    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.57:8000/ready\": dial tcp 10.134.0.57:8000: connect: connection refused"
Oct 30 21:56:30.732837 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.729382    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:56:30.757635 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.757611    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:56:30.765292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.761892    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:56:30.767576 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767554    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\" (UniqueName: \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.767679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767613    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "kube-api-access-d7x2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:30.767742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767729    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767787    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767852    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767883    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767914    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\" (UniqueName: \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\") pod \"3493f046-8d90-4b9d-80be-4560d733732b\" (UID: \"3493f046-8d90-4b9d-80be-4560d733732b\") "
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.767976    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768121    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768261    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768363    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768376    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768389    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.768426 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.768409    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7x2b\" (UniqueName: \"kubernetes.io/projected/3493f046-8d90-4b9d-80be-4560d733732b-kube-api-access-d7x2b\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.770821 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.770798    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:30.772622 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.772600    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm" pod "3493f046-8d90-4b9d-80be-4560d733732b" (UID: "3493f046-8d90-4b9d-80be-4560d733732b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.868849 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.868813    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.868867    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\" (UniqueName: \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.868897    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.868928    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\" (UniqueName: \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.868999    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869049 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869050    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869078    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\" (UniqueName: \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869099    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869116    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\" (UniqueName: \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869177    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869198    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\") pod \"6d97fa3d-1a83-4083-9368-84fcd10957f3\" (UID: \"6d97fa3d-1a83-4083-9368-84fcd10957f3\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869297    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869351    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.869783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869357    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\") pod \"2f11ec74-8136-402e-aac6-d8b7b903336a\" (UID: \"2f11ec74-8136-402e-aac6-d8b7b903336a\") "
Oct 30 21:56:30.869783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869634    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.869783 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869741    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869786    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869872    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869883    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3493f046-8d90-4b9d-80be-4560d733732b-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869892    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869900    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869908    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3493f046-8d90-4b9d-80be-4560d733732b-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869916    6738 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-tmp-dir\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.869975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869923    6738 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-model-cache\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.870417 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.869977    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.871858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.871830    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.872614 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.872590    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:30.872733 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.872716    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:30.873118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.873096    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq" pod "6d97fa3d-1a83-4083-9368-84fcd10957f3" (UID: "6d97fa3d-1a83-4083-9368-84fcd10957f3"). InnerVolumeSpecName "kube-api-access-q8bcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:30.873118 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.873110    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "kube-api-access-f784k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:30.873274 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.873259    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm" pod "2f11ec74-8136-402e-aac6-d8b7b903336a" (UID: "2f11ec74-8136-402e-aac6-d8b7b903336a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971494    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6d97fa3d-1a83-4083-9368-84fcd10957f3-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971522    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8bcq\" (UniqueName: \"kubernetes.io/projected/6d97fa3d-1a83-4083-9368-84fcd10957f3-kube-api-access-q8bcq\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971535    6738 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-home\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971547    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2f11ec74-8136-402e-aac6-d8b7b903336a-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971555    6738 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2f11ec74-8136-402e-aac6-d8b7b903336a-tls-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971563    6738 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6d97fa3d-1a83-4083-9368-84fcd10957f3-dshm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:30.971880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:30.971573    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f784k\" (UniqueName: \"kubernetes.io/projected/2f11ec74-8136-402e-aac6-d8b7b903336a-kube-api-access-f784k\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:31.041399 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.041353    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:31.267850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.267771    6738 generic.go:356] "Generic (PLEG): container finished" podID="3493f046-8d90-4b9d-80be-4560d733732b" containerID="464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06" exitCode=0
Oct 30 21:56:31.267850 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.267834    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"
Oct 30 21:56:31.268083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.267859    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" event={"ID":"3493f046-8d90-4b9d-80be-4560d733732b","Type":"ContainerDied","Data":"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"}
Oct 30 21:56:31.268083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.267892    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2" event={"ID":"3493f046-8d90-4b9d-80be-4560d733732b","Type":"ContainerDied","Data":"6b34fdf436049f88cbe03b64e423a7c94ec3f0cc919df1ea8e3b33e9e0f2d72a"}
Oct 30 21:56:31.268083 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.267908    6738 scope.go:122] "RemoveContainer" containerID="464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"
Oct 30 21:56:31.269374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.269348    6738 generic.go:356] "Generic (PLEG): container finished" podID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerID="81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d" exitCode=0
Oct 30 21:56:31.269509 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.269412    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"
Oct 30 21:56:31.269582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.269503    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" event={"ID":"6d97fa3d-1a83-4083-9368-84fcd10957f3","Type":"ContainerDied","Data":"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"}
Oct 30 21:56:31.269582 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.269546    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h" event={"ID":"6d97fa3d-1a83-4083-9368-84fcd10957f3","Type":"ContainerDied","Data":"a39b46a1f7a7c2f2eb6e9dcc4b2a4296eef4e8280cf628688a6e7476baec0ddf"}
Oct 30 21:56:31.271007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.270982    6738 generic.go:356] "Generic (PLEG): container finished" podID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerID="2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7" exitCode=0
Oct 30 21:56:31.271131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.271031    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"
Oct 30 21:56:31.271131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.271080    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" event={"ID":"2f11ec74-8136-402e-aac6-d8b7b903336a","Type":"ContainerDied","Data":"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"}
Oct 30 21:56:31.271131 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.271115    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt" event={"ID":"2f11ec74-8136-402e-aac6-d8b7b903336a","Type":"ContainerDied","Data":"fc24787e44bdfc86ca7a5e09ef517efd151d28fe57d850a6edb0088bfb2659b7"}
Oct 30 21:56:31.278838 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.278820    6738 scope.go:122] "RemoveContainer" containerID="464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"
Oct 30 21:56:31.279114 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:31.279096    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06\": container with ID starting with 464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06 not found: ID does not exist" containerID="464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"
Oct 30 21:56:31.279164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.279123    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06"} err="failed to get container status \"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06\": rpc error: code = NotFound desc = could not find container \"464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06\": container with ID starting with 464f9bb70007b44137edbf20be144397c1c996e94d8eef7d60b76fa7327a3b06 not found: ID does not exist"
Oct 30 21:56:31.279164 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.279140    6738 scope.go:122] "RemoveContainer" containerID="81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"
Oct 30 21:56:31.287355 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.287332    6738 scope.go:122] "RemoveContainer" containerID="81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"
Oct 30 21:56:31.287675 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:31.287656    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d\": container with ID starting with 81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d not found: ID does not exist" containerID="81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"
Oct 30 21:56:31.287778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.287680    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d"} err="failed to get container status \"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d\": rpc error: code = NotFound desc = could not find container \"81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d\": container with ID starting with 81ef0bf25e4b6ef7d415700984e5e596df9411a546a49999877f02eb2284a64d not found: ID does not exist"
Oct 30 21:56:31.287778 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.287705    6738 scope.go:122] "RemoveContainer" containerID="2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"
Oct 30 21:56:31.296633 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.296378    6738 scope.go:122] "RemoveContainer" containerID="2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"
Oct 30 21:56:31.297143 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:31.297122    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7\": container with ID starting with 2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7 not found: ID does not exist" containerID="2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"
Oct 30 21:56:31.297235 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.297149    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7"} err="failed to get container status \"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7\": rpc error: code = NotFound desc = could not find container \"2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7\": container with ID starting with 2495f72757297a320aabc824a6f6d4b701400a4908f806f78a4cd85ecc1f36b7 not found: ID does not exist"
Oct 30 21:56:31.299561 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.299491    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:56:31.306634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.306616    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-2-w1-a33b12-kserve-57cff9845c-k4kzt"]
Oct 30 21:56:31.320979 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.320957    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:56:31.327708 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.327669    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/simulator-w1-a33b12-kserve-77f96b885d-8vk2h"]
Oct 30 21:56:31.345793 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.345770    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:56:31.359344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:31.359324    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-a33b12/distinct-w1-a33b12-kserve-7846d889b6-m7gm2"]
Oct 30 21:56:32.046156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:32.046125    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:32.210428 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:32.210392    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:32.526754 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:32.526722    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:33.166707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:33.166674    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" path="/var/lib/kubelet/pods/2f11ec74-8136-402e-aac6-d8b7b903336a/volumes"
Oct 30 21:56:33.167209 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:33.167191    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3493f046-8d90-4b9d-80be-4560d733732b" path="/var/lib/kubelet/pods/3493f046-8d90-4b9d-80be-4560d733732b/volumes"
Oct 30 21:56:33.167728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:33.167711    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" path="/var/lib/kubelet/pods/6d97fa3d-1a83-4083-9368-84fcd10957f3/volumes"
Oct 30 21:56:33.491506 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:33.481028    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:34.575645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:34.575484    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.103457 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.099845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.324932 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.324894    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.480138 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.480101    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:56:35.480317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.480297    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" containerID="cri-o://481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219" gracePeriod=30
Oct 30 21:56:35.570195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.570159    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:35.777214 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:35.777122    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:36.182871 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:36.182839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:36.358742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:36.358703    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:36.497525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:36.497387    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:37.248058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:37.248031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:38.151792 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:38.150581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:38.409082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:38.408998    6738 patch_prober.go:29] interesting pod/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:56:38.409082 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:38.409058    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:56:39.003858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:39.003816    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:39.977060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:39.977025    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:40.748292 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.748225    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:56:40.915007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.914967    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915027    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915080    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915200 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915106    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915205    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915237    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915332    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915375 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915362    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915392    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\") pod \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\" (UID: \"c453299e-56e2-4cfe-ba3e-8c4bb86413c7\") "
Oct 30 21:56:40.915656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915581    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:40.915656 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915598    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:40.916120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915806    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:40.916120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915949    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:40.916120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915967    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:40.916120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.915982    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:40.916663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.916239    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:40.916663 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.916461    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:40.918568 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.918538    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:56:40.918693 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.918592    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:40.918765 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.918744    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:56:40.919668 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.919644    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9" pod "c453299e-56e2-4cfe-ba3e-8c4bb86413c7" (UID: "c453299e-56e2-4cfe-ba3e-8c4bb86413c7"). InnerVolumeSpecName "kube-api-access-tffw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:40.987940 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:40.987841    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017063    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017098    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tffw9\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-kube-api-access-tffw9\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017108    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017117    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017125    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.017151 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.017133    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c453299e-56e2-4cfe-ba3e-8c4bb86413c7-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:41.338045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.337953    6738 generic.go:356] "Generic (PLEG): container finished" podID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerID="481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219" exitCode=0
Oct 30 21:56:41.338045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.338023    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"
Oct 30 21:56:41.338045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.338042    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" event={"ID":"c453299e-56e2-4cfe-ba3e-8c4bb86413c7","Type":"ContainerDied","Data":"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"}
Oct 30 21:56:41.338342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.338088    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n" event={"ID":"c453299e-56e2-4cfe-ba3e-8c4bb86413c7","Type":"ContainerDied","Data":"4c946f6a9a9e9457de416646b2915b37b9eec8cfef0b8ee6c3e952a6a4e08467"}
Oct 30 21:56:41.338342 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.338112    6738 scope.go:122] "RemoveContainer" containerID="481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"
Oct 30 21:56:41.348644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.348624    6738 scope.go:122] "RemoveContainer" containerID="481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"
Oct 30 21:56:41.348943 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:41.348921    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219\": container with ID starting with 481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219 not found: ID does not exist" containerID="481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"
Oct 30 21:56:41.349010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.348955    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219"} err="failed to get container status \"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219\": rpc error: code = NotFound desc = could not find container \"481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219\": container with ID starting with 481256e75ee357d4eba7706d4f074a02532436a643912ff96b6142059d31c219 not found: ID does not exist"
Oct 30 21:56:41.358248 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.358219    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:56:41.364303 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.364273    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-3011c6-openshift-default-78b56fc845-d879n"]
Oct 30 21:56:41.809345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:41.809315    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:43.043912 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:43.043843    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:43.164141 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:43.164109    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" path="/var/lib/kubelet/pods/c453299e-56e2-4cfe-ba3e-8c4bb86413c7/volumes"
Oct 30 21:56:43.288715 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:43.288576    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:56:44.599978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:44.599943    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:44.746396 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:44.746363    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:45.183628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:45.183579    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.132085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:46.132056    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.290674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:46.290639    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.636348 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:46.636304    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:46.944467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:46.944372    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:47.503496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:47.503456    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:47.725987 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:47.725951    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:56:47.726467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:47.726404    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" podUID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" containerName="maas-api" containerID="cri-o://f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b" gracePeriod=30
Oct 30 21:56:47.880418 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:47.880345    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:48.033623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.033566    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:56:48.129152 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.129110    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:48.149081 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.149047    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\") pod \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") "
Oct 30 21:56:48.149382 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.149365    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\") pod \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") "
Oct 30 21:56:48.149618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.149603    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\" (UniqueName: \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\") pod \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") "
Oct 30 21:56:48.149748 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.149725    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") pod \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\" (UID: \"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f\") "
Oct 30 21:56:48.151006 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.150978    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca" pod "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" (UID: "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:56:48.153679 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.153615    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs" pod "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" (UID: "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:48.153812 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.153773    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls" pod "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" (UID: "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:56:48.155547 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.155525    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b" pod "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" (UID: "fc0446c6-73f2-43f5-9e6b-7a2f39126a8f"). InnerVolumeSpecName "kube-api-access-csq5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:56:48.251797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.251755    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:48.251797 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.251791    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:48.252077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.251805    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csq5b\" (UniqueName: \"kubernetes.io/projected/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-kube-api-access-csq5b\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:48.252077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.251819    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:56:48.290329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.290251    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing" containerID="cri-o://1342ce0e5bbb2133e250f8ada57cc567d574458da0a18e87f2962a179dcb4b2a" gracePeriod=25
Oct 30 21:56:48.364623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.364568    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:48.398857 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.398779    6738 generic.go:356] "Generic (PLEG): container finished" podID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" containerID="f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b" exitCode=0
Oct 30 21:56:48.399021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.398858    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"
Oct 30 21:56:48.399021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.398862    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" event={"ID":"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f","Type":"ContainerDied","Data":"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"}
Oct 30 21:56:48.399021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.398931    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s" event={"ID":"fc0446c6-73f2-43f5-9e6b-7a2f39126a8f","Type":"ContainerDied","Data":"82a766d62c6271dd1299043ecd080068b3aa0594bba43709ccf5e77ad459fa58"}
Oct 30 21:56:48.399021 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.398956    6738 scope.go:122] "RemoveContainer" containerID="f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"
Oct 30 21:56:48.410859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.410835    6738 scope.go:122] "RemoveContainer" containerID="f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"
Oct 30 21:56:48.411148 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:48.411128    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b\": container with ID starting with f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b not found: ID does not exist" containerID="f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"
Oct 30 21:56:48.411221 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.411159    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b"} err="failed to get container status \"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b\": rpc error: code = NotFound desc = could not find container \"f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b\": container with ID starting with f373e7eaa44b832018e5dcb4317007fd2d78ccf26545b6b5634f07a8bbadcf5b not found: ID does not exist"
Oct 30 21:56:48.435194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.435169    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:56:48.441290 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.441268    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-a33b12-7c7b776c64-jnv6s"]
Oct 30 21:56:48.572232 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:48.572200    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:49.019664 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:49.019633    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:49.164077 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:49.164047    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" path="/var/lib/kubelet/pods/fc0446c6-73f2-43f5-9e6b-7a2f39126a8f/volumes"
Oct 30 21:56:49.757364 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:49.757332    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:50.849143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:50.849111    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:52.051740 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:52.051680    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:52.508012 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:56:52.507976    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:56:53.588393 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:53.588359    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:54.687036 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:54.687005    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:56:54.762964 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:54.762905    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:55.884873 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:55.884845    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:56.194499 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:56.194466    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:56.950129 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:56.949256    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:57.732247 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:57.732188    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:58.291132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:58.291099    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:58.488939 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:58.488904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:56:59.364516 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:56:59.364475    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:00.150412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.150380    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:00.421060 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.420986    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:00.538653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.538622    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:57:00.539098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.539067    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy" containerID="cri-o://86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079" gracePeriod=30
Oct 30 21:57:00.677903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.677822    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:57:00.678059 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.678013    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" containerID="cri-o://95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663" gracePeriod=30
Oct 30 21:57:00.680598 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.680573    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:57:00.680771 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.680749    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy" containerID="cri-o://6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0" gracePeriod=30
Oct 30 21:57:00.688097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.688061    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:00.856624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:00.856588    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:01.756212 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:01.756177    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:02.623936 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:02.623904    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:03.215174 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:03.215140    6738 patch_prober.go:29] interesting pod/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 21:57:03.215575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:03.215209    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 21:57:03.464961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:03.464932    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.291236 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:04.291201    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:04.687135 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:04.687098    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:05.183394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.183365    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:05.861168 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.861134    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950083    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950144    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950182    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950218    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950317    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950407    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950455    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950489    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.951329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.950548    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\") pod \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\" (UID: \"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50\") "
Oct 30 21:57:05.953203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.952890    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:05.953642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.953304    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:05.953642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.953508    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:05.953814 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.953720    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:05.954390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.954111    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:05.954886 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.954866    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:05.958868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.958561    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "kube-api-access-fgzn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:05.958868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.958624    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:05.958868 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.958633    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy" pod "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" (UID: "bdc54a47-5b98-4b0d-b0f5-9628aee9ec50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:05.977242 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:05.977214    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:57:06.005097 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.005057    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:57:06.051288 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051252    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.051484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051313    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.051484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051349    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.051484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051409    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.051484 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051471    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.051735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051505    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.051735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051556    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.051735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051606    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.051735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051659    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.051735 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051713    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051729    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051745    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051783    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051812    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051857    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051897    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051919    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051940    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.052003 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.051963    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.052493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052025    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\") pod \"94af8791-9541-42c1-98ea-7e3992bb4ecf\" (UID: \"94af8791-9541-42c1-98ea-7e3992bb4ecf\") "
Oct 30 21:57:06.052493 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052062    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\") pod \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\" (UID: \"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6\") "
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052527    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052550    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052593    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052607    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052621    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052635    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.052658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052650    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052666    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgzn2\" (UniqueName: \"kubernetes.io/projected/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-kube-api-access-fgzn2\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052681    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052695    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052709    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052724    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052868    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052934    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.053027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.052950    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.053677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.053184    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.053677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.053224    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:06.053677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.053284    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.053677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.053298    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:06.054563 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.054535    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.056317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.056287    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.056602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.056571    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:06.056707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.056610    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q" pod "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" (UID: "cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6"). InnerVolumeSpecName "kube-api-access-vvh8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.056707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.056656    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "kube-api-access-c75mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.056707 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.056659    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:06.057601 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.057576    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:57:06.058449 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.058417    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy" pod "94af8791-9541-42c1-98ea-7e3992bb4ecf" (UID: "94af8791-9541-42c1-98ea-7e3992bb4ecf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:57:06.123550 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.123515    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:06.153467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153423    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153462    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153472    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153481    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153489    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153501    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153509    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153517    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153525    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153533    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153542    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvh8q\" (UniqueName: \"kubernetes.io/projected/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-kube-api-access-vvh8q\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153550    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94af8791-9541-42c1-98ea-7e3992bb4ecf-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153558    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c75mm\" (UniqueName: \"kubernetes.io/projected/94af8791-9541-42c1-98ea-7e3992bb4ecf-kube-api-access-c75mm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153568    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94af8791-9541-42c1-98ea-7e3992bb4ecf-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.153689 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.153576    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:06.511761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.511653    6738 generic.go:356] "Generic (PLEG): container finished" podID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerID="95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663" exitCode=0
Oct 30 21:57:06.511761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.511726    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" event={"ID":"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6","Type":"ContainerDied","Data":"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"}
Oct 30 21:57:06.512007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.511781    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz" event={"ID":"cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6","Type":"ContainerDied","Data":"8b983aa060e192fa796dfe6a87cef5ba1cc75ba1a69fcc1b29da71b1cdc84a48"}
Oct 30 21:57:06.512007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.511806    6738 scope.go:122] "RemoveContainer" containerID="95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"
Oct 30 21:57:06.512007 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.511735    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"
Oct 30 21:57:06.513594 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.513570    6738 generic.go:356] "Generic (PLEG): container finished" podID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerID="86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079" exitCode=0
Oct 30 21:57:06.513725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.513637    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"
Oct 30 21:57:06.513725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.513651    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" event={"ID":"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50","Type":"ContainerDied","Data":"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"}
Oct 30 21:57:06.513725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.513705    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742" event={"ID":"bdc54a47-5b98-4b0d-b0f5-9628aee9ec50","Type":"ContainerDied","Data":"331c945f175fcd796eabc9e9230adebca855fc0207334337f923fc0c84454336"}
Oct 30 21:57:06.515858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.515830    6738 generic.go:356] "Generic (PLEG): container finished" podID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerID="6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0" exitCode=0
Oct 30 21:57:06.515975 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.515941    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" event={"ID":"94af8791-9541-42c1-98ea-7e3992bb4ecf","Type":"ContainerDied","Data":"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"}
Oct 30 21:57:06.516058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.515980    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz" event={"ID":"94af8791-9541-42c1-98ea-7e3992bb4ecf","Type":"ContainerDied","Data":"11edad2935567b2a218164066e9272a00776390a3cc919a9ec2b6ab8e6c8a5f7"}
Oct 30 21:57:06.516058 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.516041    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"
Oct 30 21:57:06.525615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.525593    6738 scope.go:122] "RemoveContainer" containerID="95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"
Oct 30 21:57:06.526016 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:06.525987    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663\": container with ID starting with 95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663 not found: ID does not exist" containerID="95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"
Oct 30 21:57:06.526124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.526027    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663"} err="failed to get container status \"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663\": rpc error: code = NotFound desc = could not find container \"95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663\": container with ID starting with 95e6cb9cde2a8990dbfd95c10071559d3aff16f3f63844ff2b00a39b4421c663 not found: ID does not exist"
Oct 30 21:57:06.526124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.526051    6738 scope.go:122] "RemoveContainer" containerID="86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"
Oct 30 21:57:06.535610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.535591    6738 scope.go:122] "RemoveContainer" containerID="86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"
Oct 30 21:57:06.535985 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:06.535962    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079\": container with ID starting with 86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079 not found: ID does not exist" containerID="86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"
Oct 30 21:57:06.536098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.535988    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079"} err="failed to get container status \"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079\": rpc error: code = NotFound desc = could not find container \"86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079\": container with ID starting with 86269320ce4f1a450e53662d0d87635b32de62f9c2fe15498676edba7341d079 not found: ID does not exist"
Oct 30 21:57:06.536098 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.536005    6738 scope.go:122] "RemoveContainer" containerID="6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"
Oct 30 21:57:06.544917 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.544903    6738 scope.go:122] "RemoveContainer" containerID="6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"
Oct 30 21:57:06.545250 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:06.545199    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0\": container with ID starting with 6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0 not found: ID does not exist" containerID="6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"
Oct 30 21:57:06.545343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.545260    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0"} err="failed to get container status \"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0\": rpc error: code = NotFound desc = could not find container \"6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0\": container with ID starting with 6e6c137215f4e453f514de288c7b7752566d7bbe58f78176f52b7e798c4e8ae0 not found: ID does not exist"
Oct 30 21:57:06.547858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.547833    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:57:06.567466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.556730    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-xzkrz"]
Oct 30 21:57:06.582325 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.582083    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:57:06.585955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.585932    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-0ea1ea-openshift-default-78d5f77b9-zv742"]
Oct 30 21:57:06.599347 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.599324    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:57:06.608996 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:06.608971    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-a33b12-openshift-default-749cf497bc-87xbz"]
Oct 30 21:57:07.162859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:07.162829    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" path="/var/lib/kubelet/pods/94af8791-9541-42c1-98ea-7e3992bb4ecf/volumes"
Oct 30 21:57:07.163259 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:07.163245    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" path="/var/lib/kubelet/pods/bdc54a47-5b98-4b0d-b0f5-9628aee9ec50/volumes"
Oct 30 21:57:07.163674 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:07.163659    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" path="/var/lib/kubelet/pods/cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6/volumes"
Oct 30 21:57:07.577134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:07.576968    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:08.253691 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.253660    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:08.533525 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.533489    6738 generic.go:356] "Generic (PLEG): container finished" podID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerID="1342ce0e5bbb2133e250f8ada57cc567d574458da0a18e87f2962a179dcb4b2a" exitCode=0
Oct 30 21:57:08.533685 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.533611    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" event={"ID":"7af83fac-4b27-4d9a-89c5-6aa76508833b","Type":"ContainerDied","Data":"1342ce0e5bbb2133e250f8ada57cc567d574458da0a18e87f2962a179dcb4b2a"}
Oct 30 21:57:08.533742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.533705    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq" event={"ID":"7af83fac-4b27-4d9a-89c5-6aa76508833b","Type":"ContainerDied","Data":"a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff"}
Oct 30 21:57:08.533742 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.533735    6738 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a0cfc426e674151f76aa48ef5f57a1aaf64a6b6a163265bdba605b12e38ce5ff"
Oct 30 21:57:08.550037 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.550016    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:57:08.587143 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.587116    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\") pod \"7af83fac-4b27-4d9a-89c5-6aa76508833b\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") "
Oct 30 21:57:08.587305 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.587206    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\" (UniqueName: \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\") pod \"7af83fac-4b27-4d9a-89c5-6aa76508833b\" (UID: \"7af83fac-4b27-4d9a-89c5-6aa76508833b\") "
Oct 30 21:57:08.587495 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.587473    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume" pod "7af83fac-4b27-4d9a-89c5-6aa76508833b" (UID: "7af83fac-4b27-4d9a-89c5-6aa76508833b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:08.589566 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.589538    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk" pod "7af83fac-4b27-4d9a-89c5-6aa76508833b" (UID: "7af83fac-4b27-4d9a-89c5-6aa76508833b"). InnerVolumeSpecName "kube-api-access-h7xkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:08.688734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.688705    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7xkk\" (UniqueName: \"kubernetes.io/projected/7af83fac-4b27-4d9a-89c5-6aa76508833b-kube-api-access-h7xkk\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:08.688734 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:08.688732    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7af83fac-4b27-4d9a-89c5-6aa76508833b-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:09.050958 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:09.050921    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:09.539177 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:09.539140    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"
Oct 30 21:57:09.561595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:09.561567    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:57:09.565785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:09.565760    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-0ea1ea-6d49dc8959-tcwjq"]
Oct 30 21:57:09.954947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:09.954909    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:10.777275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:10.777221    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:11.164803 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:11.164771    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" path="/var/lib/kubelet/pods/7af83fac-4b27-4d9a-89c5-6aa76508833b/volumes"
Oct 30 21:57:11.244899 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:11.244869    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:11.448372 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:11.448265    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:11.677345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:11.677295    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:12.488153 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:12.488105    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:13.351644 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:13.351614    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.333889 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:14.333848    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.543840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:14.543800    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:14.722776 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:14.722741    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:15.110010 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:15.109924    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:15.707786 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:15.707755    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:16.609485 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:16.609419    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:17.564973 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:17.564938    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:18.390872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:18.390836    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.092949 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:19.092913    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:19.864344 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:19.864296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:20.539411 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:20.539375    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.242178 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:21.242142    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.363575 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:21.363519    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:21.947327 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:21.947296    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:22.147612 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:22.147581    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:22.505816 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:22.505788    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:22.510597 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:22.510564    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:57:22.997995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:22.997958    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:23.927089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:23.927060    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:24.770374 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:24.770343    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:25.473165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:25.473132    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:25.593744 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:25.593710    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.294725 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:26.294694    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:26.876500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:26.876469    6738 scope.go:122] "RemoveContainer" containerID="1342ce0e5bbb2133e250f8ada57cc567d574458da0a18e87f2962a179dcb4b2a"
Oct 30 21:57:27.090066 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:27.090031    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:27.885096 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:27.885049    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:28.677027 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:28.676991    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:29.363874 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:29.363839    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:30.047038 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:30.047005    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:30.769378 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:30.769326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:31.441419 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:31.441393    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:32.159329 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:32.159300    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:32.852768 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:32.852739    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.022747 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:33.022717    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:57:33.044928 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:33.044891    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:57:33.130194 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:33.130155    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.274346 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:33.274310    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:33.566832 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:33.566747    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.409981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:34.409942    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:34.939125 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:34.939094    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:57:34.939340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:34.939314    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" podUID="95571c93-227f-4acd-a730-921faa0c2cab" containerName="maas-api" containerID="cri-o://e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e" gracePeriod=30
Oct 30 21:57:35.050762 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.050718    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.256068 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.256034    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:57:35.272626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.272592    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.356253 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.356216    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\") pod \"95571c93-227f-4acd-a730-921faa0c2cab\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") "
Oct 30 21:57:35.356470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.356271    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\" (UniqueName: \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\") pod \"95571c93-227f-4acd-a730-921faa0c2cab\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") "
Oct 30 21:57:35.356470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.356374    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\") pod \"95571c93-227f-4acd-a730-921faa0c2cab\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") "
Oct 30 21:57:35.356470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.356399    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\") pod \"95571c93-227f-4acd-a730-921faa0c2cab\" (UID: \"95571c93-227f-4acd-a730-921faa0c2cab\") "
Oct 30 21:57:35.356921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.356892    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca" pod "95571c93-227f-4acd-a730-921faa0c2cab" (UID: "95571c93-227f-4acd-a730-921faa0c2cab"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:35.358687 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.358665    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls" pod "95571c93-227f-4acd-a730-921faa0c2cab" (UID: "95571c93-227f-4acd-a730-921faa0c2cab"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:35.359030 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.359007    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q" pod "95571c93-227f-4acd-a730-921faa0c2cab" (UID: "95571c93-227f-4acd-a730-921faa0c2cab"). InnerVolumeSpecName "kube-api-access-86h4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:35.359134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.359113    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs" pod "95571c93-227f-4acd-a730-921faa0c2cab" (UID: "95571c93-227f-4acd-a730-921faa0c2cab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:57:35.434246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.434217    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.457935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.457908    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:35.457935 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.457932    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86h4q\" (UniqueName: \"kubernetes.io/projected/95571c93-227f-4acd-a730-921faa0c2cab-kube-api-access-86h4q\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:35.458078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.457942    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95571c93-227f-4acd-a730-921faa0c2cab-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:35.458078 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.457951    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95571c93-227f-4acd-a730-921faa0c2cab-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:35.583100 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.583023    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:35.690721 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.690691    6738 generic.go:356] "Generic (PLEG): container finished" podID="95571c93-227f-4acd-a730-921faa0c2cab" containerID="e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e" exitCode=0
Oct 30 21:57:35.690907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.690818    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"
Oct 30 21:57:35.690907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.690839    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" event={"ID":"95571c93-227f-4acd-a730-921faa0c2cab","Type":"ContainerDied","Data":"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"}
Oct 30 21:57:35.690907 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.690900    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k" event={"ID":"95571c93-227f-4acd-a730-921faa0c2cab","Type":"ContainerDied","Data":"c34883cdadd2b2bd276a99ddc1dd182cf0e89c0114354260896cc77487ee3ab6"}
Oct 30 21:57:35.691099 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.690928    6738 scope.go:122] "RemoveContainer" containerID="e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"
Oct 30 21:57:35.700820 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.700796    6738 scope.go:122] "RemoveContainer" containerID="e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"
Oct 30 21:57:35.701319 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:35.701295    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e\": container with ID starting with e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e not found: ID does not exist" containerID="e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"
Oct 30 21:57:35.701471 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.701333    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e"} err="failed to get container status \"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e\": rpc error: code = NotFound desc = could not find container \"e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e\": container with ID starting with e472b726fba8df819b29f2ac426805e44de0627a1e3e121c20c636f786de6f5e not found: ID does not exist"
Oct 30 21:57:35.722626 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.722603    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:57:35.725373 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.725351    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6ed261-68bc7dd8b7-chs9k"]
Oct 30 21:57:35.963376 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:35.963349    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:36.912864 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:36.912827    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:37.163050 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:37.162971    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95571c93-227f-4acd-a730-921faa0c2cab" path="/var/lib/kubelet/pods/95571c93-227f-4acd-a730-921faa0c2cab/volumes"
Oct 30 21:57:37.933589 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:37.933557    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:38.023603 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:38.023562    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing" containerID="cri-o://d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb" gracePeriod=25
Oct 30 21:57:38.045933 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:38.045860    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing" containerID="cri-o://5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a" gracePeriod=25
Oct 30 21:57:38.775602 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:38.775566    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:39.554359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:39.554326    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:40.402121 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:40.402084    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:41.233659 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:41.233622    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.206833 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:42.206785    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:42.982562 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:42.982525    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:43.392233 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:43.392199    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:43.549468 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:43.549422    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:43.740808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:43.740728    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:43.879496 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:43.879424    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.377636 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:44.377600    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.824336 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:44.824240    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:44.938921 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:44.938885    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:45.893361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:45.893325    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:45.943002 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:45.942962    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:46.647450 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:46.647406    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:47.606631 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:47.606593    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:48.363615 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:48.363576    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:49.288369 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:49.288338    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:49.438785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:49.438751    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.173701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:50.173671    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:50.867738 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:50.867697    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:51.570701 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:51.570671    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:52.507608 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:52.507566    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:57:52.585753 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:52.585726    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:53.248714 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:53.248684    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:53.946466 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:53.946410    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:54.629075 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:54.629030    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:54.760507 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:54.760414    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:57:54.938858 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:54.938769    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:55.942828 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:55.942783    6738 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 21:57:58.322872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.322845    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:57:58.342539 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.342517    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:57:58.387963 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.387935    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\" (UniqueName: \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\") pod \"2be9f280-09d3-4aea-a658-ba19946d72d1\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") "
Oct 30 21:57:58.388120 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.388033    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\") pod \"2be9f280-09d3-4aea-a658-ba19946d72d1\" (UID: \"2be9f280-09d3-4aea-a658-ba19946d72d1\") "
Oct 30 21:57:58.388398 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.388375    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume" pod "2be9f280-09d3-4aea-a658-ba19946d72d1" (UID: "2be9f280-09d3-4aea-a658-ba19946d72d1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:58.390564 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.390537    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g" pod "2be9f280-09d3-4aea-a658-ba19946d72d1" (UID: "2be9f280-09d3-4aea-a658-ba19946d72d1"). InnerVolumeSpecName "kube-api-access-qlg9g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:58.489195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.489162    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\") pod \"6c0c3504-66bc-4976-a053-61ec06d5a548\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") "
Oct 30 21:57:58.489345 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.489260    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\" (UniqueName: \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\") pod \"6c0c3504-66bc-4976-a053-61ec06d5a548\" (UID: \"6c0c3504-66bc-4976-a053-61ec06d5a548\") "
Oct 30 21:57:58.489572 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.489558    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlg9g\" (UniqueName: \"kubernetes.io/projected/2be9f280-09d3-4aea-a658-ba19946d72d1-kube-api-access-qlg9g\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:58.489645 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.489580    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2be9f280-09d3-4aea-a658-ba19946d72d1-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:58.489684 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.489634    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume" pod "6c0c3504-66bc-4976-a053-61ec06d5a548" (UID: "6c0c3504-66bc-4976-a053-61ec06d5a548"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:57:58.491510 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.491490    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm" pod "6c0c3504-66bc-4976-a053-61ec06d5a548" (UID: "6c0c3504-66bc-4976-a053-61ec06d5a548"). InnerVolumeSpecName "kube-api-access-8t9bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:57:58.590595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.590504    6738 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c0c3504-66bc-4976-a053-61ec06d5a548-plugins-config-volume\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:58.590595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.590547    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t9bm\" (UniqueName: \"kubernetes.io/projected/6c0c3504-66bc-4976-a053-61ec06d5a548-kube-api-access-8t9bm\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:57:58.822414 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.822371    6738 generic.go:356] "Generic (PLEG): container finished" podID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerID="d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb" exitCode=0
Oct 30 21:57:58.822617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.822420    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"
Oct 30 21:57:58.822617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.822424    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" event={"ID":"2be9f280-09d3-4aea-a658-ba19946d72d1","Type":"ContainerDied","Data":"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"}
Oct 30 21:57:58.822617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.822510    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb" event={"ID":"2be9f280-09d3-4aea-a658-ba19946d72d1","Type":"ContainerDied","Data":"fdf484a9671d77913ac766af94af01e2b44bb32129ed675de0fb354a727f3db1"}
Oct 30 21:57:58.822617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.822539    6738 scope.go:122] "RemoveContainer" containerID="d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"
Oct 30 21:57:58.824176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.824150    6738 generic.go:356] "Generic (PLEG): container finished" podID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerID="5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a" exitCode=0
Oct 30 21:57:58.824302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.824210    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"
Oct 30 21:57:58.824302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.824226    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" event={"ID":"6c0c3504-66bc-4976-a053-61ec06d5a548","Type":"ContainerDied","Data":"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"}
Oct 30 21:57:58.824302 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.824259    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w" event={"ID":"6c0c3504-66bc-4976-a053-61ec06d5a548","Type":"ContainerDied","Data":"b971235506826ee12f4847435ad6951c88a834173ad2d9ed1297cb4b91b76af7"}
Oct 30 21:57:58.837208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.835203    6738 scope.go:122] "RemoveContainer" containerID="d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"
Oct 30 21:57:58.837208 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:58.835844    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb\": container with ID starting with d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb not found: ID does not exist" containerID="d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"
Oct 30 21:57:58.837208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.835921    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb"} err="failed to get container status \"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb\": rpc error: code = NotFound desc = could not find container \"d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb\": container with ID starting with d62781a796b5976cea8403b83fe49f8591c3d1c58a63dc5ee8b9d62be0d91ccb not found: ID does not exist"
Oct 30 21:57:58.837208 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.835945    6738 scope.go:122] "RemoveContainer" containerID="5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"
Oct 30 21:57:58.845872 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.845856    6738 scope.go:122] "RemoveContainer" containerID="5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"
Oct 30 21:57:58.846150 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:57:58.846132    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a\": container with ID starting with 5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a not found: ID does not exist" containerID="5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"
Oct 30 21:57:58.846203 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.846160    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a"} err="failed to get container status \"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a\": rpc error: code = NotFound desc = could not find container \"5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a\": container with ID starting with 5e883251dc7a6b2880bc969dd1865b7328908a516867fc2def309319b2da429a not found: ID does not exist"
Oct 30 21:57:58.855683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.855664    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:57:58.861230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.861207    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-6ed261-64f99c4cf-dgxzb"]
Oct 30 21:57:58.884223 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.884198    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:57:58.895577 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:58.895557    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-6ed261-77d747cf9f-jx64w"]
Oct 30 21:57:59.164555 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:59.164524    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" path="/var/lib/kubelet/pods/2be9f280-09d3-4aea-a658-ba19946d72d1/volumes"
Oct 30 21:57:59.164942 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:57:59.164926    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" path="/var/lib/kubelet/pods/6c0c3504-66bc-4976-a053-61ec06d5a548/volumes"
Oct 30 21:58:10.305677 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.305450    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:58:10.306161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.305717    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" podUID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" containerName="maas-api" containerID="cri-o://788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491" gracePeriod=30
Oct 30 21:58:10.380618 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.380586    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.593134 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.593106    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:58:10.705859 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.705829    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\") pod \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") "
Oct 30 21:58:10.706055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.705877    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\") pod \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") "
Oct 30 21:58:10.706055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.705966    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\" (UniqueName: \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\") pod \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") "
Oct 30 21:58:10.706055 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.706048    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\") pod \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\" (UID: \"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb\") "
Oct 30 21:58:10.706470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.706427    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca" pod "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" (UID: "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:58:10.708257 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.708229    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q" pod "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" (UID: "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb"). InnerVolumeSpecName "kube-api-access-j929q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:58:10.708371 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.708265    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls" pod "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" (UID: "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:10.708658 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.708632    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs" pod "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" (UID: "7ac0febb-a2c1-4f02-9329-fbbb7e68fccb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 21:58:10.757109 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.757082    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.807391 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.807339    6738 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-openshift-service-ca\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:10.807391 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.807391    6738 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-metrics-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:10.807578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.807430    6738 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-maas-api-tls\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:10.807578 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.807473    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j929q\" (UniqueName: \"kubernetes.io/projected/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb-kube-api-access-j929q\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:58:10.889017 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.888987    6738 generic.go:356] "Generic (PLEG): container finished" podID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" containerID="788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491" exitCode=0
Oct 30 21:58:10.889161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.889045    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"
Oct 30 21:58:10.889161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.889072    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" event={"ID":"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb","Type":"ContainerDied","Data":"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"}
Oct 30 21:58:10.889161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.889131    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n" event={"ID":"7ac0febb-a2c1-4f02-9329-fbbb7e68fccb","Type":"ContainerDied","Data":"eda96ee7a0ff0599c6e51d47272bad76265ecded7a6a0f7ddf1ff5106c46351a"}
Oct 30 21:58:10.889161 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.889159    6738 scope.go:122] "RemoveContainer" containerID="788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"
Oct 30 21:58:10.900283 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.900264    6738 scope.go:122] "RemoveContainer" containerID="788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"
Oct 30 21:58:10.900580 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:58:10.900559    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491\": container with ID starting with 788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491 not found: ID does not exist" containerID="788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"
Oct 30 21:58:10.900662 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.900592    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491"} err="failed to get container status \"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491\": rpc error: code = NotFound desc = could not find container \"788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491\": container with ID starting with 788ce54f4a331e780bdda274263728417f77534e48af589e558e72594e724491 not found: ID does not exist"
Oct 30 21:58:10.920497 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.920473    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:10.921206 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.921142    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:58:10.923390 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:10.923364    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-05a342-59b774d77b-cl92n"]
Oct 30 21:58:11.162480 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:11.162381    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" path="/var/lib/kubelet/pods/7ac0febb-a2c1-4f02-9329-fbbb7e68fccb/volumes"
Oct 30 21:58:11.188785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:11.188752    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.290699 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:17.290669    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.451637 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:17.451597    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:17.778898 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:17.778860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:19.360180 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:19.360155    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-z767s_0e5fb926-7759-4b04-9f9e-5c933d2c62d5/kuadrant-console-plugin/0.log"
Oct 30 21:58:19.655891 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:19.655859    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-nxr8f_a19267cc-024e-466e-a236-266f5c1f5f41/manager/0.log"
Oct 30 21:58:21.081176 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:21.081143    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:58:21.320595 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:21.320561    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:21.467926 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:21.467902    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:58:21.956549 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:21.956522    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:58:22.307328 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:22.307259    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:58:22.506703 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:58:22.506672    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:58:22.530840 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:22.530817    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:58:26.040923 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:26.040860    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.195853 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:26.195815    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:26.412105 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:26.412067    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:28.195089 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:28.195059    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:33.779688 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:33.779653    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:58:34.175796 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:34.175765    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:58:34.202764 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:34.202740    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:34.673294 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:34.673261    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:58:35.033359 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:35.033288    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:58:35.246334 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:35.246304    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:58:36.005052 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:36.005016    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6b56594c78-wsdp7_c4ee81fb-a3c1-4ada-b749-7ab223d1ef84/maas-api/0.log"
Oct 30 21:58:38.056893 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:38.056867    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xkl8h_6f9f2326-10fb-40e1-a2ad-772f38cc3c21/postgres/0.log"
Oct 30 21:58:40.170230 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:40.170205    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-z767s_0e5fb926-7759-4b04-9f9e-5c933d2c62d5/kuadrant-console-plugin/0.log"
Oct 30 21:58:40.444610 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:40.444523    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-nxr8f_a19267cc-024e-466e-a236-266f5c1f5f41/manager/0.log"
Oct 30 21:58:42.113503 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:42.113466    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-f7hgs_ea85e6a1-85a1-4a78-a913-705ad58be881/discovery/0.log"
Oct 30 21:58:42.674500 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:42.674407    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5978974d9d-zr6ts_56a025c6-fe78-42c0-849f-00afa1a76084/kube-auth-proxy/0.log"
Oct 30 21:58:42.845937 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:42.845900    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-46k27_2bc6e654-4219-4f28-a8df-57f51f09a730/istio-proxy/0.log"
Oct 30 21:58:43.646412 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:43.646386    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-vgrr4_b791e110-a436-4eba-9674-7c0fded22180/istio-proxy/0.log"
Oct 30 21:58:44.166683 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:44.166654    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-xcwwf_a04cf628-eece-472b-91c0-8b25c2407e57/istio-proxy/0.log"
Oct 30 21:58:52.507055 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:58:52.507009    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:58:57.537451 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:57.537403    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:58:57.657779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:58:57.657743    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:02.155124 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:02.155095    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-26bcq_8e34a640-bd21-49d6-a1de-487db5d06085/global-pull-secret-syncer/0.log"
Oct 30 21:59:02.419903 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:02.419826    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log"
Oct 30 21:59:02.594616 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:02.594580    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-2.us-east-2.compute.internal_f96f6cddab271c4d36b89844c0672b86/haproxy/0.log"
Oct 30 21:59:10.707862 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:10.707834    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-z767s_0e5fb926-7759-4b04-9f9e-5c933d2c62d5/kuadrant-console-plugin/0.log"
Oct 30 21:59:10.810586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:10.810551    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-nxr8f_a19267cc-024e-466e-a236-266f5c1f5f41/manager/0.log"
Oct 30 21:59:12.831343 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.831292    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/alertmanager/0.log"
Oct 30 21:59:12.856817 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.856786    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/config-reloader/0.log"
Oct 30 21:59:12.880617 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.880595    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/kube-rbac-proxy-web/0.log"
Oct 30 21:59:12.907165 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.907142    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/kube-rbac-proxy/0.log"
Oct 30 21:59:12.931140 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.931121    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/kube-rbac-proxy-metric/0.log"
Oct 30 21:59:12.955995 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.955973    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/prom-label-proxy/0.log"
Oct 30 21:59:12.981609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:12.981591    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3fd7b2f8-1d04-4b58-9277-983f00415c56/init-config-reloader/0.log"
Oct 30 21:59:13.173785 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.173765    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b7c7n_95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/kube-state-metrics/0.log"
Oct 30 21:59:13.276613 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.276590    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b7c7n_95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/kube-rbac-proxy-main/0.log"
Oct 30 21:59:13.367552 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.367529    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-b7c7n_95ffda81-b63a-4fbf-ae0a-79f27bc7eaf7/kube-rbac-proxy-self/0.log"
Oct 30 21:59:13.399299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.399276    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-8b68d8d9-5tfbm_d3abab98-e4d8-4ebd-b4a3-81d91079ab72/metrics-server/0.log"
Oct 30 21:59:13.428823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.428750    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-72dpx_4fa94833-b283-4190-b623-f36b7c9edf2f/monitoring-plugin/0.log"
Oct 30 21:59:13.643728 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.643699    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srnnf_2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/node-exporter/0.log"
Oct 30 21:59:13.667628 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.667602    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srnnf_2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/kube-rbac-proxy/0.log"
Oct 30 21:59:13.697557 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.697493    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-srnnf_2ce6b46c-9dba-4f64-bb0e-50287d8b5d8f/init-textfile/0.log"
Oct 30 21:59:13.730367 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.730346    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pm65s_068a733e-f3f0-4aec-9db1-a78b9e81c332/kube-rbac-proxy-main/0.log"
Oct 30 21:59:13.756779 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.756752    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pm65s_068a733e-f3f0-4aec-9db1-a78b9e81c332/kube-rbac-proxy-self/0.log"
Oct 30 21:59:13.783330 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.783306    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-pm65s_068a733e-f3f0-4aec-9db1-a78b9e81c332/openshift-state-metrics/0.log"
Oct 30 21:59:13.835634 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.835609    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/prometheus/0.log"
Oct 30 21:59:13.860775 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.860753    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/config-reloader/0.log"
Oct 30 21:59:13.896156 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.896134    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/thanos-sidecar/0.log"
Oct 30 21:59:13.922703 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.922688    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/kube-rbac-proxy-web/0.log"
Oct 30 21:59:13.949025 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.948977    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/kube-rbac-proxy/0.log"
Oct 30 21:59:13.974976 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:13.974954    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/kube-rbac-proxy-thanos/0.log"
Oct 30 21:59:14.001527 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.001509    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_e7861629-db0d-4d8b-aa64-f4ba432dd2cd/init-config-reloader/0.log"
Oct 30 21:59:14.138299 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.138271    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bff7c9d65-54l6l_dc1286b6-f07e-496f-8f52-475379c5b185/telemeter-client/0.log"
Oct 30 21:59:14.172501 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.172476    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bff7c9d65-54l6l_dc1286b6-f07e-496f-8f52-475379c5b185/reload/0.log"
Oct 30 21:59:14.199586 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.199511    6738 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"]
Oct 30 21:59:14.199978 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.199964    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.199980    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.199996    6738 cpu_manager.go:414] "containerMap: removing container" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200001    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200008    6738 cpu_manager.go:414] "containerMap: removing container" podUID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" containerName="maas-api"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200013    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" containerName="maas-api"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200020    6738 cpu_manager.go:414] "containerMap: removing container" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing"
Oct 30 21:59:14.200026 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200025    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200037    6738 cpu_manager.go:414] "containerMap: removing container" podUID="235355db-f889-4889-a406-d3e371902f52" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200045    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="235355db-f889-4889-a406-d3e371902f52" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200053    6738 cpu_manager.go:414] "containerMap: removing container" podUID="4b1a1130-125c-4344-8efc-00466623cf8b" containerName="main"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200057    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4b1a1130-125c-4344-8efc-00466623cf8b" containerName="main"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200063    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200067    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200080    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e374033d-2102-4de2-8520-28eb00acdd15" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200084    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e374033d-2102-4de2-8520-28eb00acdd15" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200089    6738 cpu_manager.go:414] "containerMap: removing container" podUID="7c7b1113-9813-4257-bb65-3b1a03ce1455" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200093    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c7b1113-9813-4257-bb65-3b1a03ce1455" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200099    6738 cpu_manager.go:414] "containerMap: removing container" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200103    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200110    6738 cpu_manager.go:414] "containerMap: removing container" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200117    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200127    6738 cpu_manager.go:414] "containerMap: removing container" podUID="95571c93-227f-4acd-a730-921faa0c2cab" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200133    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95571c93-227f-4acd-a730-921faa0c2cab" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200138    6738 cpu_manager.go:414] "containerMap: removing container" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200144    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200149    6738 cpu_manager.go:414] "containerMap: removing container" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200153    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200163    6738 cpu_manager.go:414] "containerMap: removing container" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200167    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200174    6738 cpu_manager.go:414] "containerMap: removing container" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200178    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200186    6738 cpu_manager.go:414] "containerMap: removing container" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200191    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200201    6738 cpu_manager.go:414] "containerMap: removing container" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200208    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200215    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200219    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200226    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200231    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200237    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200241    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200248    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200253    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200258    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerName="maas-api"
Oct 30 21:59:14.200249 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200263    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200272    6738 cpu_manager.go:414] "containerMap: removing container" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200280    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200288    6738 cpu_manager.go:414] "containerMap: removing container" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200293    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200302    6738 cpu_manager.go:414] "containerMap: removing container" podUID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200306    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200312    6738 cpu_manager.go:414] "containerMap: removing container" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200316    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200323    6738 cpu_manager.go:414] "containerMap: removing container" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200327    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200333    6738 cpu_manager.go:414] "containerMap: removing container" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200336    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200343    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200347    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200358    6738 cpu_manager.go:414] "containerMap: removing container" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200365    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200372    6738 cpu_manager.go:414] "containerMap: removing container" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200376    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200381    6738 cpu_manager.go:414] "containerMap: removing container" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200386    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200396    6738 cpu_manager.go:414] "containerMap: removing container" podUID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200400    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200407    6738 cpu_manager.go:414] "containerMap: removing container" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200411    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200419    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200423    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200446    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200452    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200461    6738 cpu_manager.go:414] "containerMap: removing container" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200467    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200473    6738 cpu_manager.go:414] "containerMap: removing container" podUID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200478    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200484    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200489    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200495    6738 cpu_manager.go:414] "containerMap: removing container" podUID="670ad8df-c084-4e8a-bb8f-6de1017cb331" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200499    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="670ad8df-c084-4e8a-bb8f-6de1017cb331" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200504    6738 cpu_manager.go:414] "containerMap: removing container" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200508    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200514    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200518    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200525    6738 cpu_manager.go:414] "containerMap: removing container" podUID="2a990500-0296-4a9f-8fd0-7cf50282841e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200532    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a990500-0296-4a9f-8fd0-7cf50282841e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200543    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200548    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200556    6738 cpu_manager.go:414] "containerMap: removing container" podUID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200561    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200573    6738 cpu_manager.go:414] "containerMap: removing container" podUID="280474f1-ae85-4cbb-81e0-955a58c482b6" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200610    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="280474f1-ae85-4cbb-81e0-955a58c482b6" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200625    6738 cpu_manager.go:414] "containerMap: removing container" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200631    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200643    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200649    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200661    6738 cpu_manager.go:414] "containerMap: removing container" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200668    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200678    6738 cpu_manager.go:414] "containerMap: removing container" podUID="49f7d339-a578-4f54-b05c-3a8f5709b17d" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200684    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49f7d339-a578-4f54-b05c-3a8f5709b17d" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200701    6738 cpu_manager.go:414] "containerMap: removing container" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200707    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200721    6738 cpu_manager.go:414] "containerMap: removing container" podUID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200728    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200739    6738 cpu_manager.go:414] "containerMap: removing container" podUID="98068c9f-8e54-414b-aa66-73aae297f5c1" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200745    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98068c9f-8e54-414b-aa66-73aae297f5c1" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200755    6738 cpu_manager.go:414] "containerMap: removing container" podUID="baa40822-2e8f-4f91-8a17-3febe3a476fc" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200762    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="baa40822-2e8f-4f91-8a17-3febe3a476fc" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200776    6738 cpu_manager.go:414] "containerMap: removing container" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200783    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200793    6738 cpu_manager.go:414] "containerMap: removing container" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200799    6738 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200943    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3493f046-8d90-4b9d-80be-4560d733732b" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200955    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="753953a8-b9be-4e08-a259-6d990fde1708" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200966    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0b8f042-e5c6-41c1-805e-63835541c2f3" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.200974    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="bff9b43a-2ac7-4a33-8bc4-ef6e4d4fafe4" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201007    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="9efd6bd5-49f0-44e3-9805-c8e437cc4951" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201015    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad50ef83-4035-47e7-a027-b1e6395290a2" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201024    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="235355db-f889-4889-a406-d3e371902f52" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201031    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="4b1a1130-125c-4344-8efc-00466623cf8b" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201039    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b558560-ef9f-4ebc-b90c-3c350fc1bd35" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201048    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c7b1113-9813-4257-bb65-3b1a03ce1455" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201056    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e374033d-2102-4de2-8520-28eb00acdd15" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201083    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="bdc54a47-5b98-4b0d-b0f5-9628aee9ec50" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201091    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc09f4b3-0436-41fa-ad66-86721bcee130" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201098    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="95571c93-227f-4acd-a730-921faa0c2cab" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201105    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201113    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="724f2199-282c-48c3-8e22-0075d8543f7a" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201122    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a8e6aa2-6089-43c4-a030-e0bef25f5ccc" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201129    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="321c2bf4-2dc7-4980-adc5-6d3abc6a8588" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201136    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="94af8791-9541-42c1-98ea-7e3992bb4ecf" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201162    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="c453299e-56e2-4cfe-ba3e-8c4bb86413c7" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201171    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c0c3504-66bc-4976-a053-61ec06d5a548" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201180    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2be9f280-09d3-4aea-a658-ba19946d72d1" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201189    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3126c2f-abf7-414c-904a-cc39332c3e84" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201200    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b67cdf4-bfe7-4b2d-8eee-8e6339a6b836" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201210    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4405c11-ab67-41d8-81ee-0d9690c725bd" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201235    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="817ed678-747a-494d-979e-10c0170ef7ca" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201247    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="76714185-024e-496e-b444-f9d187dedf65" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201257    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="7ac0febb-a2c1-4f02-9329-fbbb7e68fccb" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201264    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="7af83fac-4b27-4d9a-89c5-6aa76508833b" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201280    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="09aeb068-df15-48d0-9ec4-17a6558620a0" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201287    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbf52a31-a847-4ee2-9fcd-9c0ecb274750" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201294    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a5b54311-99ea-4e34-bde6-870e8cfe6a2b" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201315    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="f3ba5b05-aeac-4956-8360-f688e2aa8d55" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201322    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f5e1c6-18ae-4d6a-aedb-1bbe002a97de" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201330    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="74e13002-b907-4fad-ae17-2ff178c1550c" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201337    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d70d105-da83-4c2e-aab7-bca9dc599b5f" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201345    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc75253b-5d97-4da6-8fad-8fcd8245639a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201354    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b708ac00-44da-416f-bb61-291ab5148f74" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201388    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7c81aeb-8918-4fb7-8e4e-0cce70d6c11e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201396    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf66057a-5a1e-45b0-9f1e-84adaaa6e8a6" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201404    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc0446c6-73f2-43f5-9e6b-7a2f39126a8f" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201410    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f11ec74-8136-402e-aac6-d8b7b903336a" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201418    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="670ad8df-c084-4e8a-bb8f-6de1017cb331" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201426    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="b76a6d50-5d68-437f-87de-34a4edc6ce98" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201451    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a1cf896-b64d-4adb-9da1-c0de35fa842c" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201459    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a990500-0296-4a9f-8fd0-7cf50282841e" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201468    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c9c4cb8-4c2b-4426-8825-a2ba74e2d2ee" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201477    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="eda11f2c-5e04-418b-ab69-54cb6b5b1c7a" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201487    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="280474f1-ae85-4cbb-81e0-955a58c482b6" containerName="payload-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201496    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d97fa3d-1a83-4083-9368-84fcd10957f3" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201505    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0a4e5ca-e7f4-4ba9-a603-5dd0e1fa3e03" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201510    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3638232-08bf-420e-b13b-12cb6dce0fd7" containerName="istio-proxy"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201515    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="49f7d339-a578-4f54-b05c-3a8f5709b17d" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201521    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="428bc8c4-362d-43af-91ed-69c6e01ee773" containerName="payload-pre-processing"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201526    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="23a8f0b7-2960-4dad-a3f6-dc884a011bbe" containerName="maas-api"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201532    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="98068c9f-8e54-414b-aa66-73aae297f5c1" containerName="main"
Oct 30 21:59:14.201918 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.201537    6738 memory_manager.go:365] "RemoveStaleState removing state" podUID="baa40822-2e8f-4f91-8a17-3febe3a476fc" containerName="maas-api"
Oct 30 21:59:14.206079 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.204868    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.213781 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.213751    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzjln\"/\"openshift-service-ca.crt\""
Oct 30 21:59:14.213880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.213770    6738 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzjln\"/\"kube-root-ca.crt\""
Oct 30 21:59:14.217830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.214870    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5bff7c9d65-54l6l_dc1286b6-f07e-496f-8f52-475379c5b185/kube-rbac-proxy/0.log"
Oct 30 21:59:14.217830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.217269    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"]
Oct 30 21:59:14.217830 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.217525    6738 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qzjln\"/\"default-dockercfg-cbx5l\""
Oct 30 21:59:14.278394 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.278356    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2n6r\" (UniqueName: \"kubernetes.io/projected/0dadbd62-00ab-4ca5-a784-07819dcff42b-kube-api-access-z2n6r\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.278607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.278486    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-podres\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.278607 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.278559    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-sys\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.278808 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.278785    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-lib-modules\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.278944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.278924    6738 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-proc\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.279425 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.279407    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/thanos-query/0.log"
Oct 30 21:59:14.311340 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.311316    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/kube-rbac-proxy-web/0.log"
Oct 30 21:59:14.346246 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.346214    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/kube-rbac-proxy/0.log"
Oct 30 21:59:14.378841 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.378820    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/prom-label-proxy/0.log"
Oct 30 21:59:14.380201 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380179    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-lib-modules\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380296 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380215    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-proc\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380299    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-lib-modules\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380325    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2n6r\" (UniqueName: \"kubernetes.io/projected/0dadbd62-00ab-4ca5-a784-07819dcff42b-kube-api-access-z2n6r\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380358 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380343    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-proc\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380380    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-podres\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380429    6738 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-sys\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380460    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-podres\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.380554 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.380513    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0dadbd62-00ab-4ca5-a784-07819dcff42b-sys\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.391759 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.391735    6738 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2n6r\" (UniqueName: \"kubernetes.io/projected/0dadbd62-00ab-4ca5-a784-07819dcff42b-kube-api-access-z2n6r\") pod \"perf-node-gather-daemonset-cwhzz\" (UID: \"0dadbd62-00ab-4ca5-a784-07819dcff42b\") " pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.413085 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.413066    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/kube-rbac-proxy-rules/0.log"
Oct 30 21:59:14.441981 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.441961    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-756469fbcb-g9zw7_b78874e1-9201-4cfa-871f-1af75327111b/kube-rbac-proxy-metrics/0.log"
Oct 30 21:59:14.520308 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.520257    6738 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:14.795020 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:14.794994    6738 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"]
Oct 30 21:59:14.797129 ip-10-0-139-2 kubenswrapper[6738]: W1030 21:59:14.797092    6738 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0dadbd62_00ab_4ca5_a784_07819dcff42b.slice/crio-7bb58cbb1f79a756474009fb59163d667633c15f2deb16277ed5342a7dd22c45 WatchSource:0}: Error finding container 7bb58cbb1f79a756474009fb59163d667633c15f2deb16277ed5342a7dd22c45: Status 404 returned error can't find the container with id 7bb58cbb1f79a756474009fb59163d667633c15f2deb16277ed5342a7dd22c45
Oct 30 21:59:15.212387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:15.212349    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz" event={"ID":"0dadbd62-00ab-4ca5-a784-07819dcff42b","Type":"ContainerStarted","Data":"acdbe2af67bdc02d6fc8c91aa3a8122979c0665de3c97727c3000d9e2aaacc72"}
Oct 30 21:59:15.212804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:15.212410    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz" event={"ID":"0dadbd62-00ab-4ca5-a784-07819dcff42b","Type":"ContainerStarted","Data":"7bb58cbb1f79a756474009fb59163d667633c15f2deb16277ed5342a7dd22c45"}
Oct 30 21:59:15.212804 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:15.212462    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:15.229054 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:15.229003    6738 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz" podStartSLOduration=1.228991249 podStartE2EDuration="1.228991249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 21:59:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 21:59:15.227415579 +0000 UTC m=+3412.183102807" watchObservedRunningTime="2026-10-30 21:59:15.228991249 +0000 UTC m=+3412.184678476"
Oct 30 21:59:17.781275 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:17.781243    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:17.782123 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:17.782106    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:18.781195 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:18.781158    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:18.784132 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:18.784113    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:19.277004 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:19.276969    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:19.277627 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:19.277608    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:19.330410 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:19.330372    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:59:19.330623 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:19.330597    6738 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerName="istio-proxy" containerID="cri-o://11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5" gracePeriod=30
Oct 30 21:59:20.496892 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.496860    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:20.499045 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.499026    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:20.517142 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.517122    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:20.519476 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.519429    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:20.565470 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.565425    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jmdvh_7ad08418-9a79-4912-831e-d49f5802f695/dns/0.log"
Oct 30 21:59:20.591880 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.591861    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jmdvh_7ad08418-9a79-4912-831e-d49f5802f695/kube-rbac-proxy/0.log"
Oct 30 21:59:20.619315 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:20.619296    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-6gb6m_c2020321-9a3d-4786-b6cb-f00ad50011be/dns-node-resolver/0.log"
Oct 30 21:59:21.229008 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:21.228982    6738 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qzjln/perf-node-gather-daemonset-cwhzz"
Oct 30 21:59:21.370317 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:21.370284    6738 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-m474j_60535067-d426-468e-ab28-84675493491d/konnectivity-agent/0.log" limit=1024
Oct 30 21:59:21.551041 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:21.550969    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkbt2_4d19a8f3-e037-4087-8d58-f1e05facef69/node-ca/0.log"
Oct 30 21:59:21.968311 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:21.968283    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:22.082961 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:22.082914    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:22.514625 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:59:22.514584    6738 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-v6xnw" containerName="extractor"
Oct 30 21:59:22.936553 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:22.936523    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:23.055876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:23.055844    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:23.444245 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:23.444193    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:23.591823 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:23.591793    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:24.657761 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.657723    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:59:24.752467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752397    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.752467 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752463    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.752802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752495    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.752802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752625    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.752802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752674    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:24.752802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752756    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.752802 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752786    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752814    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752838    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752843    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.752881    6738 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\") pod \"2bc6e654-4219-4f28-a8df-57f51f09a730\" (UID: \"2bc6e654-4219-4f28-a8df-57f51f09a730\") "
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.753242    6738 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-certs\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.753273 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.753263    6738 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-workload-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.753767 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.753629    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 21:59:24.754368 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.754335    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:24.754642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.754594    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:24.755322 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.755300    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 21:59:24.756282 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.756260    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 21:59:24.756405 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.756273    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "kube-api-access-trgmx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:24.757726 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.757700    6738 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token" pod "2bc6e654-4219-4f28-a8df-57f51f09a730" (UID: "2bc6e654-4219-4f28-a8df-57f51f09a730"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 21:59:24.853876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853850    6738 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2bc6e654-4219-4f28-a8df-57f51f09a730-istiod-ca-cert\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.853876 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853876    6738 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-token\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.854136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853886    6738 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-podinfo\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.854136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853895    6738 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-credential-socket\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.854136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853904    6738 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-envoy\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.854136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853917    6738 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trgmx\" (UniqueName: \"kubernetes.io/projected/2bc6e654-4219-4f28-a8df-57f51f09a730-kube-api-access-trgmx\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:24.854136 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:24.853932    6738 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2bc6e654-4219-4f28-a8df-57f51f09a730-istio-data\") on node \"ip-10-0-139-2.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 21:59:25.100184 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.100089    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:25.160653 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.160623    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:25.227397 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.227373    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:25.270321 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.270289    6738 generic.go:356] "Generic (PLEG): container finished" podID="2bc6e654-4219-4f28-a8df-57f51f09a730" containerID="11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5" exitCode=0
Oct 30 21:59:25.270486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.270324    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" event={"ID":"2bc6e654-4219-4f28-a8df-57f51f09a730","Type":"ContainerDied","Data":"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"}
Oct 30 21:59:25.270486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.270375    6738 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"
Oct 30 21:59:25.270486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.270388    6738 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27" event={"ID":"2bc6e654-4219-4f28-a8df-57f51f09a730","Type":"ContainerDied","Data":"e25627793ff70f6954a9e654a6500feefd61dda4a48aee50a34bc54a16132263"}
Oct 30 21:59:25.270486 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.270417    6738 scope.go:122] "RemoveContainer" containerID="11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"
Oct 30 21:59:25.282944 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.282920    6738 scope.go:122] "RemoveContainer" containerID="11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"
Oct 30 21:59:25.283294 ip-10-0-139-2 kubenswrapper[6738]: E1030 21:59:25.283268    6738 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5\": container with ID starting with 11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5 not found: ID does not exist" containerID="11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"
Oct 30 21:59:25.283380 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.283307    6738 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5"} err="failed to get container status \"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5\": rpc error: code = NotFound desc = could not find container \"11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5\": container with ID starting with 11a02098bbb8055ba379e2d0f58679ecbe0571d4fa7759d673d84a45da010cc5 not found: ID does not exist"
Oct 30 21:59:25.287977 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.287961    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:25.291624 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.291604    6738 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:59:25.297970 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:25.297945    6738 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-75bf867b6c-46k27"]
Oct 30 21:59:26.856947 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:26.856907    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:27.021053 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:27.021023    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-7tfz5_01001045-1325-4307-b938-8418266b3ec0/manager/0.log"
Oct 30 21:59:27.164890 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:27.164861    6738 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2bc6e654-4219-4f28-a8df-57f51f09a730" path="/var/lib/kubelet/pods/2bc6e654-4219-4f28-a8df-57f51f09a730/volumes"
Oct 30 21:59:28.048587 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:28.048549    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:28.248955 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:28.248923    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-hzj4m_6b4f1153-0be9-417b-bd15-8cdb5bdaa169/manager/0.log"
Oct 30 21:59:28.688931 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:28.688896    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:28.878145 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:28.878109    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-c55679bf5-2hd6c_ab742809-f65e-4524-aae4-bc1714674dda/manager/0.log"
Oct 30 21:59:29.482387 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:29.482336    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-f7hgs_ea85e6a1-85a1-4a78-a913-705ad58be881/discovery/0.log"
Oct 30 21:59:30.317361 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.317315    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:30.407731 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.407687    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:30.446642 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.446616    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-769875db56-s4vkj_796c707d-027e-4303-aa99-6b46d94485d6/manager/0.log"
Oct 30 21:59:30.520000 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.519975    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-s2cll_e53aa576-fca2-4a2c-944d-cf252d4f239c/proxy/0.log"
Oct 30 21:59:30.690609 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.690582    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5978974d9d-zr6ts_56a025c6-fe78-42c0-849f-00afa1a76084/kube-auth-proxy/0.log"
Oct 30 21:59:30.866402 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.866361    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-vgrr4_b791e110-a436-4eba-9674-7c0fded22180/istio-proxy/0.log"
Oct 30 21:59:30.945966 ip-10-0-139-2 kubenswrapper[6738]: I1030 21:59:30.945865    6738 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-75bf867b6c-xcwwf_a04cf628-eece-472b-91c0-8b25c2407e57/istio-proxy/0.log"